Impact of Cross-Domain Intermediate Tasks on Zero-Shot Cross-Lingual Transfer Robustness in XTREME-R

Transfer learning from large language models (LLMs) has emerged as a powerful technique to enable knowledge-based fine-tuning for a number of tasks, adaptation of models for different domains and even languages. However, it remains an open question, if and when transfer learning will work, i.e. leading to positive or negative transfer. In this paper, we analyze the knowledge transfer across three natural language processing (NLP) tasks - text classification, sentimental analysis, and sentence similarity, using three LLMs - BERT, RoBERTa, and XLNet - and analyzing their performance, by fine-tun

Research goal: What is the impact of using intermediate tasks from different linguistic domains (e.g., natural language inference vs. question answering) on the robustness of zero-shot cross-lingual transfer performance in the XTREME-R benchmark when the target language belongs to a different language family than the intermediate task language?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.7/10.