DigiTala's rating criteria: Holistic and analytic scales for assessing L2 speaking

DigiTala's rating criteria: Holistic and analytic scales for assessing L2 speaking

DigiTala research project developed one holistic and five analytic rating scales for assessing L2 learners speech samples. The 2020 versions were used during the Swedish rating round and 2021 version during Finnish rating rounds. The human ratings were used for training an automated scoring system.

The raters recruited by the project assessed L2 Swedish and Finnish learners' speech samples online using Moodle (version 3.8.3). In addition to the criteria, the platform included assessment guidelines and benchmark samples illustrating different proficiency levels. An hour-long rater training was organized using Zoom. 

The rating criteria were developed for research purposes. As a starting point level descriptors of the previous upper secondary school curriculum (National Core Curriculum 2003) were used. In the previous curriculum speaking is described in a more detailed and analytic way which suits better for automated assessment purposes. The 2003 level descriptors are a Finnish application of the Common European Framework of Reference. 

More information: Kautonen, M. & von Zansen, A. (2020). DigiTala research project: Automatic speech recognition in assessing L2 speaking. Kieli, koulutus ja yhteiskunta, 11(4). https://www.kieliverkosto.fi/fi/journals/kieli-koulutus-ja-yhteiskunta-kesakuu-2020/digitala-research-project-automatic-speech-recognition-in-assessing-l2-speaking