Structural Instability Precedes Behavioral Failure in Neural Networks: Evidence from Recovery-Time Inflation and Layer-wise Dynamics

Neural network training can fail abruptly and unpredictably, with loss divergence or catastrophic instability appearing without clear early warning in standard behavioral metrics. We investigate whether structural instability exists as a measurable precursor to such failures.

We introduce Recovery-Time Inflation (RTI) as an operational early-warning observable, defined as a normalized measure of proximity to the stability boundary of the training operator and computed via Hessian spectral estimation.

Across two experiments—an MLP on a synthetic classification task and a four-layer transformer on a synthetic grammar task—we show that RTI consistently detects impending collapse before loss divergence occurs. In the MLP, RTI leads behavioral failure in 8 of 10 runs, while gradient variance provides no early warning. In the transformer, RTI leads in all 6 of 6 runs and further reveals that instability is spatially structured, originating in early layers and propagating through depth before any global behavioral signal appears.

These results demonstrate that neural network failure is both temporally preceded and structurally organized, and that RTI provides a practical, architecture-aware early-warning signal grounded in spectral dynamics.