Multilingual Intermediate-Task Training for Cross-Lingual NLI Accuracy on XNLI

We present Unicoder, a universal language encoder that is insensitive to different languages. Given an arbitrary NLP task, a model can be trained with Unicoder using training data in one language and directly applied to inputs of the same task in other languages. Comparing to similar efforts such as Multilingual BERT and XLM, three new cross-lingual pre-training tasks are proposed, including cross-lingual word recovery, cross-lingual paraphrase classification and cross-lingual masked language model. These tasks help Unicoder learn the mappings among different languages from more perspectives.

Research goal: How does multilingual intermediate-task training compare to English-only training on cross-lingual natural language inference accuracy using the XNLI benchmark?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.9/10.