Contrastive Learning for Improved Word Alignment in Multilingual BERT Embeddings

Cross-language pre-trained models such as multilingual BERT (mBERT) have achieved significant performance in various cross-lingual downstream NLP tasks. This paper proposes a multi-level contrastive learning (ML-CTL) framework to further improve the cross-lingual ability of pre-trained models. The proposed method uses translated parallel data to encourage the model to generate similar semantic embeddings for different languages. However, unlike the sentence-level alignment used in most previous studies, in this paper, we explicitly integrate the word-level information of each pair of parallel

Research goal: How does contrastive learning improve word alignment in multilingual BERT embeddings for zero-shot cross-lingual text classification on the XTREME benchmark?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.7/10.