Comparing LLM-Generated Tips and Expert-Created Tips in Quantum Computing Education

This dataset includes the anonymized data from two studies with the goal to evaluate if LLM-generated tips can be used instead of expert-created tips to help students answer quantum computing questions.
For the main study (main_test_anonymous.csv) a between-subject design was used to quizz participants of the QUIKSTART 2024 summer school, giving them four multiple-choice quantum physics questions and one tip per question. Each participant was assigned one of four conditions represented by the combination of "creator" and "labeled_as" column in the csv file.  In addition we asked to rate quality, correctness and helpfulness for each tip and to rate the perceived difficulty of the question. We removed demographics for anonymities sake.

Additionally, we conducted a study directly comparing the LLM-generated and expert-created tips (tip_eval_anonymous.csv). Where we let experts and students rate the tips helpfulness, correctness, if they gave away the answer and if they pointed to relevant concepts. Furthermore, participants had to decide for each question which tip they preferred and were able to leave a comment to give their reasoning. We removed demographics for anonymities sake.

These datasets were evaluated in the paper "LLM-Generated Tips Rival Expert-Created Tips in Helping Students Answer Quantum-Computing Questions" by Lars Krupp, Jonas Bley, Isacco Gobbi, Alexander Geng, Sabine Müller, Sungho Suh, Ali Moghiseh, Arcesio Castaneda Medina, Valeria Bartsch, Artur Widera, Herwig Ott, Paul Lukowicz, Jakob Karolus, Maximilian Kiefer-Emmanouilidis