Impact of Scaling Synthetic Data on Zero-Shot Multilingual Dense Retrieval Accuracy in SWIM-IR

Zero-shot evaluation of information retrieval (IR) models is often performed using BEIR; a large and heterogeneous benchmark composed of multiple datasets, covering different retrieval tasks across various domains. Although BEIR has become a standard benchmark for the zero-shot setup, its exclusively English content reduces its utility for underrepresented languages in IR, including Dutch. To address this limitation and encourage the development of Dutch IR models, we introduce BEIR-NL by automatically translating the publicly accessible BEIR datasets into Dutch. Using BEIR-NL, we evaluated a

Research goal: What is the impact of scaling synthetic training data volume in SWIM-IR on the zero-shot retrieval accuracy of multilingual dense retrieval models evaluated on the BEIR benchmark compared to models trained on fixed-size human-annotated datasets?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.5/10.