Cross-lingual Transfer Performance Variation with Intermediate-Task Dataset Size on PAWS-X

Multi-lingual language models (LM), such as mBERT, XLM-R, mT5, mBART, have been remarkably successful in enabling natural language tasks in low-resource languages through cross-lingual transfer from high-resource ones. In this work, we try to better understand how such models, specifically mT5, transfer *any* linguistic and semantic knowledge across languages, even though no explicit cross-lingual signals are provided during pre-training. Rather, only unannotated texts from each language are presented to the model separately and independently of one another, and the model appears to implicitly

Research goal: What is the impact of varying the size of the intermediate-task training dataset on the cross-lingual transfer performance of models on PAWS-X, measured by F1 score across high- and low-resource language pairs?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.7/10.