Subword Tokenization Strategies in Cross-Lingual Retrieval for Kinyarwanda: mBERT Versus Monolingual RoBERTa on XTREME

Pre-trained multilingual language models (e.g., mBERT, XLM-RoBERTa) have significantly advanced the state-of-the-art for zero-shot cross-lingual information extraction. These language models ubiquitously rely on word segmentation techniques that break a word into smaller constituent subwords. Therefore, all word labeling tasks (e.g. named entity recognition, event detection, etc.), necessitate a pooling strategy that takes the subword representations as input and outputs a representation for the entire word. Taking the task of cross-lingual event detection as a motivating example, we show that

Research goal: What is the impact of subword tokenization strategies on the cross-lingual retrieval accuracy of mBERT versus monolingual RoBERTa variants for Kinyarwanda passages in the XTREME dataset?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.3/10.