Entropy-Modulated Temperature Scaling in Softmax: A Self-Regulating Causal Mechanism

Standard softmax functions are "brittle," collapsing to near-absolute certainty at low temperatures. This paper investigates a self-regulating alternative by making temperature scaling a function of the system's own Shannon Entropy ($H$). We demonstrate that this entropy-modulation, $\tau_{\text{eff}} = \tau_{\text{base}} \cdot (1 + H(p))$, acts as a powerful damping mechanism. In a toy model ($n=1000$ samples, 5 classes), this mechanism produced three starkly different effects from standard softmax: (1) it resisted "collapse" by reducing maximum probability by 15.2 percentage points at low temperature ($\tau=0.1$); (2) it "remembered" unlikely options, increasing minimum probability 94-fold; and (3) it reached 89\% of maximum entropy at a moderate temperature ($\tau=0.52$), proving its self-regulating nature. These results provide an empirical "proof of principle" for a stable, non-brittle causal system, demonstrating a mathematical mechanism that avoids the paradoxical instability of simpler, self-referential loops.

v0.1 Disclaimer: This is an early draft intended to share data and demonstrate proof-of-concept. AI-assisted writing was used, and manual revisions are forthcoming. Think of this as beta access—open for inspection, not yet ready for prime time