Effects of Scenario-Based Assessment on Students' Writing Processes

This study investigates the effects of a scenario-based assessment design on students' writing processes. An experimental data set consisting of four design conditions was used in which the number of scenarios (one or two) and the placement of the essay task with respect to the lead-in tasks (first vs. last) were varied. Students' writing processes on the essay task were recorded using keystroke logs. Each keystroke action was classified into one of four writing states: planning, text production, local edit, or jump edit, and a semi-Markov model was fit to the data. Results showed that the single-scenario and essay-last design encouraged fewer but longer editing states compared to the alternative designs. Additionally, this task ordering appeared to have enabled more fluent and efficient text production when paired with a sin- gle scenario. These results seem explainable from cognitive writing theory, particularly with respect to working memory load. Limitations and future directions for research are also discussed.