Perceptual Learning Modules (PLM) in CS1: a Negative Result and a Methodological Warning

Perceptual Learning Modules (PLMs) is a variation of Perceptual Learning based on multiple-choice questionnaires. There exists successful research of the use of PLMs in math and flight training. The possibility of designing and adopting PLMs in Introductory Programming Courses (CS1) is still an open area of study. The goal of this study is to test whether students that received a PLM training on recognising segments of programs will perform better at writing programs. Two PLM interventions were administered to students. The first intervention was a nonrandom controlled experiment, in which students opted to answer the PLM questionnaire (N=40), while the control group consisted of students that did not answer it (N=629). The second intervention was a randomized controlled experiment with a placebo, in which students were randomly assigned to perform either the PLM questionnaire (N=51) or another a placebo activity (N=51). The different forms of analysis of the first experiment results yielded Cohen's d ranging from 0.23 to 0.34 in favor of the PLM intervention. For the second experiment, the effect size was d = -0.11 against the PLM intervention, but the two results were significant. We believe that the cautious conclusion is that there is a null effect in using a PLM activity as part of a CS1 course. The paper is also of interest because of the methodological decisions and techniques used.