Empirical Dynamical Centers of Transformer Generation: A Geometric Framework and Its Implications for Alignment
Description
We present SERC (Structure–Energy–Resonance–Coherence), a geometric framework for monitoring language model generation dynamics in real time. The system state is mapped to the standard 3-simplex via four components derived from hidden states. A relational tension functional Ω measures distance from the simplex center P₀. We prove that P₀ is the unique globally asymptotically stable equilibrium of the projected gradient flow, with exponential relaxation rate e^{−32t}. A series of eight experiments (NB1–NB8) across three models yields four empirical contributions: (1) R-dominance correlations of 0.84–0.94; (2) C as the primary diagnostic dimension; (3) a guardrail achieving TP=1, FP=0 with 89.5% token reduction — the first content-free pathology detection based solely on generation geometry; (4) a fundamental finding: geometric P₀ is not the natural center of any tested model's dynamics. Empirical barycenters lie far from (0.25, 0.25, 0.25, 0.25), are stable across prompt sets, and constitute a measurable property of model architecture and training. RLHF training moves the barycenter further from geometric P₀ — not closer — by systematically increasing C-dominance. This raises the question: is the empirical barycenter a computable characterization of a model, and can it predict alignment behavior?
Files
SERC_v2_0_publikacja.pdf
Files
(2.6 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:40024beace977ee0c84802716d3ba736
|
179.8 kB | Preview Download |
|
md5:c49585f986eaf2383b61139a751b9ae9
|
18.3 kB | Preview Download |
|
md5:7753b2e7e871aa85fbe3d5421cca322a
|
117.3 kB | Preview Download |
|
md5:53226c376c8f4bb935310fcc1453a3b9
|
21.9 kB | Preview Download |
|
md5:3813c2f465b678b60b4784ec2babeb52
|
637.5 kB | Preview Download |
|
md5:db172009af165f2167fa9ff0c26b6950
|
608.5 kB | Preview Download |
|
md5:12198661b02251c1c171b324701ea38f
|
223.5 kB | Preview Download |
|
md5:4402c96079ebb5caa97c3c000c24bab8
|
313.2 kB | Preview Download |
|
md5:942c9c2f37a668acfa985a0093491ac5
|
451.5 kB | Preview Download |
Additional details
Related works
- References
- Software: 10.5281/zenodo.18685546 (DOI)