Zero-Shot Cross-Lingual Retrieval: SWIM-IR Versus Synthetic Multilingual FAQ Training on Non-English BEIR Benchmarks

Zero-shot evaluation of information retrieval (IR) models is often performed using BEIR; a large and heterogeneous benchmark composed of multiple datasets, covering different retrieval tasks across various domains. Although BEIR has become a standard benchmark for the zero-shot setup, its exclusively English content reduces its utility for underrepresented languages in IR, including Dutch. To address this limitation and encourage the development of Dutch IR models, we introduce BEIR-NL by automatically translating the publicly accessible BEIR datasets into Dutch. Using BEIR-NL, we evaluated a

Research goal: How does the zero-shot cross-lingual retrieval performance of multilingual dense retrievers trained on SWIM-IR compare to those trained on synthetic multilingual FAQ datasets when evaluated on non-English BEIR benchmarks like BEIR-NL?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 7.9/10.