Supervisory Feedback vs. AI: A Comparative Study on Postgraduate Student Satisfaction

Abstract

Feedback plays a crucial role in postgraduate research supervision, influencing students’ academic progress and satisfaction. Traditional supervisory feedback is valued for its engagement and contextual relevance, while artificial intelligence (AI)-generated feedback, particularly from models like ChatGPT, is gaining attention for its clarity and accessibility. However, limited quantitative research has explored students’ comparative perceptions of AI versus human feedback. This study examines how postgraduate students perceive ChatGPT-generated feedback compared to traditional supervisory feedback. Specifically, it evaluates feedback clarity, relevance, accuracy, consistency, and comprehensiveness. Additionally, it investigates how different versions of ChatGPT (3.5 vs. 4) influence students’ satisfaction with both AI-generated and supervisor-provided feedback. A cross-sectional study was conducted with 169 postgraduate students (M.A. and Ph.D.), who completed a structured questionnaire assessing their experiences with both feedback sources. Data were analyzed using independent t-tests to compare feedback perception across clarity, relevance and accuracy, and consistency and comprehensiveness. Supervisor feedback was also evaluated on engagement. Normality tests were performed before statistical analysis to ensure validity. Findings indicate that ChatGPT’s feedback was rated higher in clarity than supervisor feedback, but lower in relevance, accuracy, and consistency. Supervisor feedback was perceived as more engaging and contextually appropriate. Users of ChatGPT 4 reported higher satisfaction with AI-generated feedback compared to ChatGPT 3.5 users, while supervisor feedback satisfaction remained consistent across both groups. Notably, reliance on AI for feedback increased as AI performance improved. AI-generated feedback offers advantages in accessibility and clarity but lacks the engagement and contextual awareness of human supervision. While AI tools can supplement academic guidance, they should not replace the critical thinking, personalized mentorship, and nuanced evaluation provided by human supervisors. The findings highlight the need for a balanced approach that integrates AI-driven feedback with traditional supervisory engagement to optimize postgraduate learning experiences.

Keywords: Academic supervision, artificial intelligence, ChatGPT, feedback quality, postgraduate research, student perceptions.