Cross-lingual Transfer Performance of mT5 on XTREME-R with Intermediate Tasks from Different Linguistic Typologies

Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, whereby it enables effective zero-shot cross-lingual transfer of syntactic knowledge. The transfer is more successful between some languages, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages. In this work, we investigate the distributions of grammatical relations induced from mBERT in the context of 24 typologically different languages. We demonstrate that the distance between the distributions of different languages is highly consistent

Research goal: Does the zero-shot cross-lingual transfer performance of mT5 on XTREME-R vary when using intermediate tasks from different linguistic typologies (e.g., SVO vs. SOV languages), as measured by accuracy differences across target languages with distinct syntactic structures?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.1/10.