Finding More, Fusing Less: A Pre-Registered Locator Bake-off for AI Agents

Companion study to "Reading More, Finding Less: A Pre-Registered Anatomy of Progressive Disclosure for AI Agents" (doi:10.5281/zenodo.21960138).

The parent study attributed a curated disclosure policy's failure to mis-routing by its index, but left the winning arm's own miss ceiling — associative search failing to reach the gold document on 19.6–25% of questions — unattributed. This companion tests whether that ceiling is reducible by ranked lexical retrieval, under predictions frozen before any test question existed.

On the parent's frozen 141-document public corpus, with a fresh 115-question set generated under mechanically enforced gates, four locator arms ran behind an identical k=8 tool contract. BM25 with per-language analyzers located the gold document on 93.0% of questions against hardened exact search's 71.3% (+21.7 pp; McNemar p < 1e-5): the miss ceiling was substantially a ranking problem. The primary registered prediction passed (+13.9 pp for the fused arm over exact search, p = 0.00014, threshold +5.0). A second frozen prediction — that reciprocal rank fusion never costs localization against its own best leg — failed: fusion trailed BM25-alone by 7.8 pp (p = 0.0225), and the pre-registered ablation rule retired the shipped fusion default automatically. An off-the-shelf SQLite FTS5 control (89.6%) was not statistically distinguishable from the custom implementation. Downstream accuracy moved +5.2 pp, decomposing into +7.5 pp from localization partly offset by −2.3 pp of conditional accuracy — the parent's anatomy replicated on a third, blind question set. Verdict: revised, reported with the passing and failing predictions at equal prominence.

The findings were independently validated before writing: an isolated re-computation session reproduced 21 of 22 claimed numbers exactly from the raw records and confirmed the verdict; an isolated literature session live-verified all 13 cited works.

This deposit contains the report (PDF and source), the frozen harness and analyzers, the gated question set with generation log and candidate cache, all run records, the mechanical adjudicator and its output, the registration and correction lineage, the corpus sha256 manifest (the snapshot itself is published with the parent deposit), and both validation session kits, returns, and their disposition — everything required to re-run the study.

GitHub mirror: https://github.com/mentu-ai/agent-memory-allocation (c36-locator-bakeoff/)
Contact: rashid@mentu.ai