AI Supported IELTS Preparation With Human Adherence Support Under Equal Planned Study Time: A Four Arm Stratified Randomized Trial

Background: AI-supported language learning is increasingly offered alongside established forms of IELTS preparation, but randomized comparisons of complete preparation services remain limited.

Objective: To compare AI-supported digital preparation with non-instructional human adherence support (AI plus support) against self-study, large-group classes, and one-to-one tutoring under equal planned study time.

Methods: In this four-arm, individually randomized, stratified trial, 960 adults with estimated entry IELTS Academic bands of 4.5–6.0 were allocated equally to four preparation formats for 40 planned hours over 8 weeks. In the AI arm, instruction, practice, feedback, and question answering were delivered through the Yingwu Zhixue (Parrot Learning; 鹦鹉智学) platform, comprising a mobile app and a companion web interface; human staff provided study supervision only. The primary outcome was a four-skill mock-test composite averaged across two post-test forms and adjusted for the average of two pre-test forms. Writing and speaking were independently scored by masked raters. The intention-to-treat analysis used multiple imputation and accounted for clustering within classes, tutors, and adherence-support staff caseloads. Three prespecified comparisons used Bonferroni-adjusted 98.33% confidence intervals, providing at least 95% simultaneous coverage. The primary criterion required all three lower bounds to exceed zero.

Results: Post-test completion was 82.4% (791/960). Adjusted differences favored AI plus support over self-study (0.54 composite-score units, 98.33% CI 0.33–0.76), large-group classes (0.45, 0.20–0.71), and one-to-one tutoring (0.45, 0.21–0.69); all Bonferroni-adjusted p values were < .001. Complete-case estimates were similar. Satisfaction estimates favored AI plus support over self-study and large-group classes; the comparison with one-to-one tutoring did not establish superiority or equivalence.

Conclusions: Under equal planned study time, assignment to AI plus support improved mock-test performance relative to the three comparison services as implemented in this trial. The findings concern the combined service and do not establish AI-specific effects, per-hour efficiency, or gains in official IELTS scores.

Funding: The study was funded by Zhiying Technology (Hangzhou) Co., Ltd., which also provides the product evaluated in the AI plus support arm.