Detecting and Repairing Coherence Failures in Long-Horizon AI Agents

We introduce CRepair, a benchmark for measuring structural self-repair in AI agents operating over long task horizons. Current evaluation frameworks assess whether agents stay coherent; CRepair measures whether agents can detect, diagnose, and recover from their own coherence failures. We define a composite metric Cₛₑₚₐᵢᵣ = D × R × V × S (all components ∈ [0,1]) and a taxonomy of six failure types, culminating in the novel repair-induced cascade category. Running Claude Sonnet across 52 scenario-condition pairs, we find that detection, repair, and verification are dissociable capabilities requiring distinct architectural interventions: a reflection prompt raises detection from 54% to 92% but leaves verification near zero; a memory prompt maximises detection (100%) but suppresses repair (46%); only an explicit repair loop raises verification substantially (0% → 38%), eliminates cascade failures entirely, and yields 5/13 perfect Cₛₑₚₐᵢᵣ = 1.0 scores. These results suggest that verification — not detection — is the binding constraint on AI self-repair, and that closing the detect–repair–verify loop requires explicit architectural support beyond prompting alone.