Consistency Is All You Need — Empirical Validation of a Control-Field Architecture
Authors/Creators
Description
Large language models often suffer from long-horizon instability, including repetitive degeneration and compounding internal inconsistency. While these behaviors are widely observed, they are typically addressed through decoding heuristics or post-hoc filtering rather than architectural intervention.
This work explores consistency as an explicit, trainable internal control signal within Transformer architectures. We introduce a lightweight control-field mechanism (CF-HoT) that modulates attention updates based on a learned estimate of internal instability, acting as a bounded negative-feedback system during forward passes.
The contribution of this release is architectural validation rather than performance claims. Phase A demonstrates that models augmented with a control field can be trained end-to-end at nontrivial scale without numerical instability or optimization collapse. Phase B shows that the same mechanism can be attached as a lightweight adapter to a frozen 8B language model, where it measurably influences generation behavior after minimal training and reduces repetitive degeneration on matched prompts.
No claims are made regarding general reasoning improvement, benchmark superiority, or production readiness. These results establish feasibility and mechanism-level effect, motivating further controlled ablations and independent replication.
Code, configuration details, and limitations are provided to support scrutiny and follow-on work.
Files
phase_b_validation_essay_v2.pdf
Files
(183.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:21f1c0d4f6c5e15ae54c1cbc589ce6c0
|
183.2 kB | Preview Download |