StaleBench: A Benchmark for Answer Freshness in Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) systems can store a new fact in the index and still answer with the old one. Most freshness checks look at the index, not at the answer the user actually receives, so this gap is rarely measured.

StaleBench measures answer freshness: after a fact changes, how long a RAG system takes to give the new value, and how often it keeps giving the old one. It reports catch-up latency and a stale-answer rate across common refresh policies, with confidence intervals, and scores answers by exact match, so no language model acts as a judge. It runs as a black-box tool against any RAG system whose documents can be changed and refreshed.

Experiments with open models show that a notable share of answers stay stale even when the index is refreshed at once, and that a simple change to how retrieved documents are ordered reduces the problem. StaleBench lets teams measure this gap in their own systems.