How to Catch Novice Programmers' Struggles: Detecting Moments of Struggle in Open-Ended Block-Based Programming Projects using Trace Log Data

Regardless of skill level and background, programming can be challenging for all students. However, in the early stages of learning, challenges may particularly lead to a decrease in students’ sense of self-efficacy and interest in computer science. Hence, finding the moments when novices struggle during programming will help us provide support and intervene at the proper time. Some efforts have been made to find out when students struggle during a specific assignment, but none of them (to our knowledge) have targeted open-ended tasks, i.e., tasks that have no fixed solutions or processes to reach the objective. This study aims to determine how students’ coding traces in a block-based programming environment relate to their struggles while completing an open-ended project. We ran a study in an introductory programming course that used a block-based language for the first 8 weeks of a semester-long class, culminating in a 2-week-long project. Students were given class time for two sessions to work on their projects in pairs, during which we collected students’ coding traces. Based on experts’ hypotheses and two prior studies, we developed detectors to parse coding traces and find struggle moments automatically. We also conducted a survey at the end of each session to ask students about their satisfaction with their programming and feelings when encountering programming challenges for which we defined detectors. We investigated how well moments identified by our detectors associated with students’ immediate survey responses. Our results show that students’ perceptions of the experienced challenges significantly correlate with detectable patterns in their coding traces.