Prefix-Tuning vs. Adapter-Based Fine-Tuning in Zero-Shot Cross-Lingual Generation on XTREME-R

With the release of new large language models (LLMs) like Llama and Mistral, zero-shot cross-lingual transfer has become increasingly feasible due to their multilingual pretraining and strong generalization capabilities. However, adapting these decoder-only LLMs to new tasks across languages remains challenging. While parameter-efficient fine-tuning (PeFT) techniques like Low-Rank Adaptation (LoRA) are widely used, prefix-based techniques such as soft prompt tuning, prefix tuning, and Llama Adapter are less explored, especially for zero-shot transfer in decoder-only models. We present a compre

Research goal: How does the performance of prefix-tuning compare to adapter-based fine-tuning in zero-shot cross-lingual generation across diverse language families beyond African languages in the XTREME-R benchmark?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 9.0/10.