RankArena Multi-Perspective Evaluation for Hallucination Detection in Cross-Domain Retrieval Versus Single-Metric Baselines
Description
Evaluating the quality of retrieval-augmented generation (RAG) and document reranking systems remains challenging due to the lack of scalable, user-centric, and multi-perspective evaluation tools. We introduce RankArena, a unified platform for comparing and analysing the performance of retrieval pipelines, rerankers, and RAG systems using structured human and LLM-based feedback as well as for collecting such feedback. RankArena supports multiple evaluation modes: direct reranking visualisation, blind pairwise comparisons with human or LLM voting, supervised manual document annotation, and end-
Research goal: To what extent does RankArena's multi-perspective evaluation framework improve the detection of hallucination failure modes in cross-domain retrieval tasks compared to single-metric baselines?
Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 8.7/10.
Notes
Files
paper.pdf
Files
(76.6 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:01d989478341f69a357ee131db877d8e
|
76.6 kB | Preview Download |