StaleBench: A Benchmark for Answer Freshness in Retrieval-Augmented Generation

StaleBench measures answer freshness in Retrieval-Augmented Generation (RAG) Systems, after a fact changes, how long a system keeps giving the old answer instead of the new one. It reports catch-up latency & recovery rate across refresh policies (never, batch, immediate), scored by exact match against a controlled ground truth (no language model as a judge), and runs on any RAG system whose documents can be changed.

Across ten open models from three families (Qwen, Llama, Gemma; released 2024 to 2026), about half of all answers stay stale even with immediate re-indexing; the cause is the position of the documents in the context, not the retrieval. Placing the newest document last fully fixes it for the most capable models but backfires on others, so the fix must be checked per system.

New in this version
Expanded the study from 3 to 10 models, added sparse/dense/reranked retrieval and a competence control, tightens the statistics (n = 144), and refines the central result into a model-dependent fix with direct evidence of the position bias behind it.

For more info, check the repository link attached.