Explore-Consolidate Dynamics in Cross-Probe Coherence Separate Successful and Failed LLM Agent Trajectories
Description
We propose cross-probe coherence κ_t — the mean absolute pairwise correlation of N concurrent per-turn behavioral probes within a moving window of agent turns — as a meta-signal for LLM agent monitoring. On 99 SWE-bench Pro trajectories from Qwen3.6-27B we report two findings. (1) Per-trace mean κ̄ separates success from failure at AUROC 0.677 (Mann-Whitney p=0.0009). (2) κ_t exhibits a U-shape over each trajectory: it falls through an early exploration phase and rises through a late consolidation phase, and the AMPLITUDE of the U is markedly larger in successful traces (early-half slope p=0.0002, late-half slope p=0.00004). The pattern is the inverse of cardiac uncoupling: in ICU literature, cross-vital decorrelation anticipates decompensation; in LLM agents, the cross-probe trajectory OSCILLATES during successful reasoning and stays flat during failure. A pre-registered robustness control caught a substantial trace-length confound in an earlier monolithic-slope version of the headline; the U-shape decomposition is the post-control rescue and is length-normalized by construction. Five pre-registered single-probe candidates were walked back before this finding emerged; we document this walk-back-and-rescue trajectory explicitly as the source of methodological credibility.
Notes
Files
fig8_early_late_half.png
Additional details
Related works
- Is identical to
- Working paper: https://openinterp.org/research/papers/kappa-t-coherence-buildup (URL)
- Is supplement to
- Software: https://github.com/OpenInterpretability/openinterp-swebench-harness (URL)
- Is supplemented by
- Dataset: https://huggingface.co/datasets/caiovicentino1/openinterp-kappa-t-coherence-buildup (URL)