StaleBench: A Benchmark for Answer Freshness in Retrieval-Augmented Generation

StaleBench measures answer freshness in Retrieval-Augmented Generation (RAG) Systems, after a fact changes, how long a system keeps giving the old answer instead of the new one. It reports catch-up latency & recovery rate across refresh policies (never, batch, immediate), scored by exact match against a controlled ground truth (no language model as a judge), and runs on any RAG system whose documents can be changed.

Across ten open models from three families (Qwen, Llama, Gemma; released 2024 to 2026), about half of all answers stay stale even with immediate re-indexing; the cause is the position of the documents in the context, not the retrieval. It also finds the cause, document position, and tests a fix (placing the newest document last) that attempts to solve it for capable models but backfires on others, so each system must be measured rather than assumed.