Modality matters in measuring foreign language aptitude: Empirical evidence from a new aptitude battery Tests of Aptitude for Language Learning (TALL)

Foreign language aptitude is a componential construct conceptualised to predict and explain second language (L2) learning outcomes (Carroll, 1981). L2 learning, across naturalistic and instructed contexts, occurs in different modalities including auditory and written forms. This entails that aptitude components―like associative memory, language analytic ability, and executive control capacity in working memory― should perhaps be operationalised using stimuli in both aural and written modalities. Thus, aptitude assessments would ideally involve tests in both modalities to reflect different learning contexts. However, most major aptitude batteries only include written tests, or do not specify the modality of test items. The selection of modality in operationalising aptitude goes beyond methodological concerns: it holds theoretical significance in understanding the role aptitude may play in diverse learning contexts.

Design and methods. This study addresses the neglected issue of modality and its impact on the outcomes of aptitude tests. A new internet-based aptitude battery—Tests of Aptitude for Language Learning (TALL; Pan & Marsden, under review)—was developed. TALL integrates constructs from the theoretical frameworks of the Stages Approach (Skehan, 2016) and the Phonological/Executive Model (Wen, 2016) and its design is informed by previous aptitude batteries. It operationalises aptitude components representing: associative memory; phonetic coding ability; language analytic ability; and working memory (specifically, phonological short-term memory and executive control capacity). These components are measured by five subtests: Vocabulary Learning, Sound Discrimination, Language Analysis, Serial Nonwords Recall, and Complex Span Tasks, respectively.

Critically, TALL has both an aural and a written ‘suite’. To allow us to both examine within-subject effects of modality and to control for order of testing, each suite contains two versions of materials with counterbalanced test items.

165 college-level L1 Chinese-L2 English participants (aged 17 ‒ 19) took two rounds of TALL tests: half taking the oral modality first and half the written modality, with a minimum 30-day interval before taking the other modality. TALL, designed to reduce confounds with L2 proficiency, underwent scrutiny that demonstrated high levels of internal validity as a battery for measuring the components theorised to constitute the aptitude construct. Mixed-effects modelling was used to analyse data from the three subtests that were administered in two modalities: Vocabulary Learning, Language Analysis, and Complex Span Tasks.

Findings. Participants consistently performed better when tested in the written modality compared to the aural modality. Furthermore, correlations of the scores from the two modalities were low (ranging between 0.16 to 0.32, in Kendall’s tau). Importantly, the effects of modality varied in their impact on results for learners at different levels of aptitude abilities, suggesting that higher aptitude abilities may mitigate the effects of modality. Together, these results suggested substantial differences in aptitude constructs when measured in different modalities.

The study underscores the importance of considering modality as a moderating variable when examining aptitude in L2 learning, especially for learners with different aptitude levels. Additionally, it highlights that research could fruitfully investigate the role of modality-specific aptitude components in different learning contexts that may foreground the aural modality (naturalistic contexts) or written modality (instructed contexts).