Comparison of Direct Preference Optimization and Reinforcement Learning for Cross-Lingual Generative Information Retrieval

Generative information retrieval (GenIR) is a promising neural retrieval paradigm that formulates document retrieval as a document identifier (docid) generation task, allowing for end-to-end optimization toward a unified global retrieval objective. However, existing GenIR models suffer from token-level misalignment, where models trained to predict the next token often fail to capture document-level relevance effectively. While reinforcement learning-based methods, such as reinforcement learning from relevance feedback (RLRF), aim to address this misalignment through reward modeling, they intro

Research goal: How does direct preference optimization compare to reinforcement learning from human feedback in reducing token-level misalignment for cross-lingual generative information retrieval?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10.