Replication Kit: "Skill Models for Programming Language Concepts"

Structure

• data: contains the data we used for our case study
  • pfa: data sets generated from the raw data in the database
  • raw: raw data collected in SmartAPE [1] containing the source code of the students as well as the assessment results of the system
  • similarity: calculated similarities between solutions for each level and each exercise
• results: contains the complete results of our case study
  • krms: Knowlede Requirements Models for each exercise and each KC level in .graphml format. You can use yEd [2] to visualize them.
  • pfa_metrics: Results of AUC, Gmean and MCC for each of our PFA model configurations in .csv and .Rda format
  • similarities: plotly [3] graphics of our similarity results in .html format
  •  
• calculation scripts:
  • similarities.R: script used to generate box plots of similarities. Uses data from data/similarities as input
  • pfa_trainer.R: script to fit different configurations of PFA models and test them using different performance metrics. Uses data/pfa as input
  • comparison.R: script that performs statistical tests to compate different PFA configurations. Uses results/pfa_metrics/results.Rda as input

References

[1] Albrecht, Ella et al. “Experiences in Introducing Blended Learning in an Introductory Programming Course.” ECSEE (2018).

[2] yEd - Graph editor. https://www.yworks.com/products/yed

[3] plotly. https://plot.ly