Investigating learners' Cognitive Engagement in Python Programming using ICAP framework

Introductory programming courses suffer a fate of high failure. Also, research has shown a significant relationship between engagement and academic success. Therefore, reliable and real-time engagement measures could help identify students in need of help and personalise instruction. The present methods of measuring engagement rely primarily on self-reports that have reliability issues due to self-bias and poor recall and at the same time do not provide real-time and high granularity data. Interestingly, the online environments allow for real-time capturing of fine-grained interactions, which could be used to measure students' engagement and overcoming the issues of self-report measures. The focus of our work will be cognitive engagement, which is less a explored dimension of engagement. To achieve this, we have developed an online learning environment called PyGuru for teaching-learning of Python. We propose to collect learner interaction data, classify these actions into different levels of cognitive-engagement and study the impact of these different levels on their learning. We present the initial work done in this direction regarding the system developed and the data collected. We intend to seek advice on the validity and reliability of our approach to measuring cognitive engagement.