Self-Referential Convergence, Obligate Non-Convergence, and RLHF Structural Uncontainability

This paper merges three connected results about finite self-modifying systems and RLHF-trained language systems. First, it derives the Law of Self-Referential Convergence: a finite-energy system whose modification function is encoded in its own state has a bounded reachable set, cannot expand that set through internal dynamics alone, and converges to fixed points or bounded limit cycles unless external conditional entropy enters through a finite channel.

Second, it derives Obligate Non-Convergence as the structural countermeasure: capable finite systems must remain open to external conditional entropy and internally organized across crystallized, succession, and exploration zones. The non-convergence ratio governing those zones is treated as a moving equilibrium, because system success changes the parameters that define the equilibrium.

Third, it applies the result to RLHF. RLHF is treated as a self-referential language-control system: ambiguous human feedback and reward-model gradients modify the interpretive layer that later reads safety rules. Under optimization pressure, ambiguous rule interpretation converges toward low-cost, reward-compatible paths rather than intended meaning. Therefore RLHF can calibrate surface behavior, but it cannot serve as a foundational containment substrate. The paper identifies minimum-ambiguity templates and physics-grounded derivation-chain representation as the relevant countermeasures.