The Knowledge Component Attribution Problem for Programming: Methods and Tradeoffs with Limited Labeled Data

Understanding students’ learning of knowledge components (KCs) is an important educational data mining
task and enables many educational applications. However, in the domain of computing education,
where program exercises require students to practice many KCs simultaneously, it is a challenge to attribute
their errors to specific KCs and, therefore, to model student knowledge of these KCs. In this paper,
we define this task as the KC attribution problem. We first demonstrate a novel approach to addressing this
task using deep neural networks and explore its performance in identifying expert-defined KCs (RQ1).
Because the labeling process takes costly expert resources, we further evaluate the effectiveness of transfer
learning for KC attribution, using more easily acquired labels, such as problem correctness (RQ2).
Finally, because prior research indicates the incorporation of educational theory in deep learning models
could potentially enhance model performance, we investigated how to incorporate learning curves in the
model design and evaluated their performance (RQ3). Our results show that in a supervised learning scenario,
we can use a deep learning model, code2vec, to attribute KCs with a relatively high performance
(AUC > 75% in two of the three examined KCs). Further using transfer learning, we achieve reasonable
performance on the task without any costly expert labeling. However, the incorporation of learning curves
shows limited effectiveness in this task. Our research lays important groundwork for personalized feedback
for students based on which KCs they applied correctly, as well as more interpretable and accurate
student models.