Effects of AI-based educational interventions on students' metacognitive outcomes: A meta-analysis and systematic review with measurement paradigm as a theoretical moderator

The central problem concerning the effects of artificial intelligence (AI)-based educational tools on students’ metacognitive outcomes is not solely whether AI improves metacognition on average, but which layer of metacognitive evidence is used to measure this outcome. Existing syntheses have dissolved metacognition into self-regulated learning clusters and have not treated measurement paradigm as a theoretically meaningful moderator. The present study offers a two-tier evidence synthesis encompassing 183 empirical studies published between 2010 and 2026: a meta-analysis drawing on 30 independent study clusters (REML random-effects model) and a complementary narrative synthesis conducted within the Synthesis Without Meta-analysis (SWiM) framework covering 153 studies. The pre-registered primary moderator is the measurement paradigm of metacognitive outcomes across three categories: self-report, judgment/calibration, and performance/log-based indicators. The meta-analytic average effect is positive and significant; however, very high heterogeneity and a prediction interval crossing zero indicate that this average cannot be read as a context-independent “AI effect.” Moderator analyses did not reach statistical significance; nonetheless, the descriptive contrast of approximately.89 units between self-report (g =.985) and judgment/calibration (g =.092) measures is theoretically noteworthy and is structurally confounded with intervention type comparisons. In the subset analysis restricted to studies with low risk-of-bias judgements, the pooled estimate decreases to g =.265 (p = .095), revealing that the effect is sensitive to methodological quality and paradigm composition. The findings shift the field’s central question from “does AI improve metacognition?” to “which AI design supports which layer of metacognitive evidence under which pedagogical conditions?”