Intermediate-task training efficiency vs. standalone instruction-tuning for zero-shot XTREME-R performance

Instruction tuning has emerged as a powerful technique, significantly boosting zero-shot performance on unseen tasks. While recent work has explored cross-lingual generalization by applying instruction tuning to multilingual models, previous studies have primarily focused on English, with a limited exploration of non-English tasks. For an in-depth exploration of cross-lingual generalization in instruction tuning, we perform instruction tuning individually for two distinct language meta-datasets. Subsequently, we assess the performance on unseen tasks in a language different from the one used f

Research goal: How does the efficiency of intermediate-task training with instruction-tuning compare to standalone instruction-tuning for zero-shot cross-lingual performance on XTREME-R, measured by F1 score per training step?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10.