Label-aware Multi-level Contrastive Learning for Zero-shot Cross-lingual Transfer in Flan-T5 on XNLI

Cross-language pre-trained models such as multilingual BERT (mBERT) have achieved significant performance in various cross-lingual downstream NLP tasks. This paper proposes a multi-level contrastive learning (ML-CTL) framework to further improve the cross-lingual ability of pre-trained models. The proposed method uses translated parallel data to encourage the model to generate similar semantic embeddings for different languages. However, unlike the sentence-level alignment used in most previous studies, in this paper, we explicitly integrate the word-level information of each pair of parallel

Research goal: Does label-aware multi-level contrastive learning improve the zero-shot cross-lingual transferability of Flan-T5 models on the XNLI benchmark when evaluated across different language pairs?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.8/10.