Structural Collapse as Information Loss: The Exponential Decay Mechanism under Accumulating Constraints

Large language models exhibit abrupt reasoning collapse when structural contradictions accumulate in their context---a phenomenon distinct from gradual degradation under factual noise. Through controlled experiments (11 models, 5 vendors, 4B--70B+ parameters), we characterize this collapse and find: (i) multiplicative interaction between context margin (μ) and structural contradiction (δ)---two individually non-lethal stressors become lethal in combination, ruling out additive degradation models; (ii) a dimensional structure in δ where, in models with sufficient baseline accuracy, a single structural contradiction causes more damage than 10 factual contradictions; and (iii) a cushion mechanism where RLHF alignment absorbs contradictions until overwhelmed, producing a system-specific critical threshold δ_c.

These patterns are explained by the first moment method from combinatorics. Defining cumulative information loss δ = Σ I(constraint_i) in nats, we show that the multiplicative survival potential S = N_eff · (μ/μ_c) · e^(−δ) captures both the LLM collapse patterns and, in Boolean satisfiability (SAT) where δ is exactly computable, yields parameter-free predictions: the decay rate ratio α_XOR/α_random = ln 2 / |ln(7/8)| = 5.19× is an upper bound confirmed experimentally (observed: 5.04 ± 0.25, CV = 5%; the systematic shortfall is consistent with inter-constraint correlations reducing effective information loss---overlapping constraints create redundancy, so that a single violation's impact on the overall system is mitigated). The exponential penalty e^(−δ) is not an approximation but a mathematical identity with the first moment of the SAT solution count. Its contribution is an operational methodology---measuring structural conflict as information loss in nats via the first moment correspondence---validated in two controlled domains. A systematic δ = 0 control experiment (6 models, 4 vendors, 500–128,000 tokens) confirms the complementary prediction: under zero structural contradiction, models with sufficient baseline capability (μ) maintain perfect accuracy regardless of context length, while an 8B-parameter model plateaus at 36%, confirming the multiplicative structure S = μ · e^(−δ). This establishes that the phenomenon commonly attributed to "context rot" is driven by contradiction accumulation (δ > 0), not by context length itself. Extension to the second moment method (Paley–Zygmund inequality) with pair correlation function g(β) = 3/4 + (1/8)(1−β)³ quantitatively explains 74% of the gap between the first-moment upper bound (α ≈ 5.19) and the true 3-SAT threshold (α ≈ 4.27); the remaining 26% is attributable to replica symmetry breaking. All results---including the second moment overlap decomposition---are machine-verified in Lean 4 (16 modules, 160 propositions, no sorry, no axiom).