Zero-shot cross-lingual transfer performance of multilingual intermediate-task fine-tuning versus multitask fine-tuning on

A large body of recent work highlights the fallacies of zero-shot cross-lingual transfer (ZS-XLT) with large multilingual language models. Namely, their performance varies substantially for different target languages and is the weakest where needed the most: for low-resource languages distant to the source language. One remedy is few-shot transfer (FS-XLT), where leveraging only a few task-annotated instances in the target language(s) may yield sizable performance gains. However, FS-XLT also succumbs to large variation, as models easily overfit to the small datasets. In this work, we present a

Research goal: How does the zero-shot cross-lingual transfer performance of multilingual intermediate-task fine-tuning compare to multitask fine-tuning on XTREME-R when evaluated across high-, mid-, and low-resource languages?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 9.0/10.