System Prompt Framing Induces Attention-Dependent Entropy Regime Switching in Transformer Token Generation
Description
We present a systematic study of how system prompt framing modulates token-level Shannon entropy
during language model inference. Across three experimental phases totaling 3,830 inference runs, we
orthogonalize two prompt factors—relational presence (R), which frames generation as collaborative
inquiry, and epistemic openness (E), which licenses uncertainty and exploration—and measure their
individual and joint effects on next-token probability distributions. Phase 1 (900 runs across 6 models at 3
parameter scales) establishes that the co-creative condition (R+E+) produces elevated entropy relative
to controls, with effect sizes scaling from negligible at 0.5B parameters (d= 0.13, NS) to large at 7B+
(d>1.0) across four architectures. Phase 2 (2,000 runs, 8 conditions, 5 architectures) decomposes the
effect via a 2×2 factorial: the R×E interaction is superadditive on two of three responsive transformers
(+0.19, +0.21) but absent in a pure state-space model (Falcon3-Mamba-7B;−0.04), consistent with
the hypothesis that cross-factor binding requires the attention mechanism. Appending safety constraint
language to the co-creative frame suppresses the effect on transformers (d = 0.85–1.22) but not on
the SSM (d = 0.22, NS). Phase 3 (930 runs, 5-factor response surface) confirms agonist-antagonist
classifications on a single model. Baseline and analytical controls are statistically indistinguishable across
all architectures (d<0.10). We release all data, code, and prompts.
Files
vasquez_2026_entropy_regime_switching.pdf
Files
(534.8 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:80323fe7f35d35c31ab0d44a3a9d0dd1
|
534.8 kB | Preview Download |
Additional details
Related works
- Is supplement to
- Software: https://github.com/templetwo/phase-modulated-attention (URL)
- Preprint: https://osf.io/9hbtk (URL)
- Is supplemented by
- Technical note: 10.5281/zenodo.19377909 (DOI)
Software
- Repository URL
- https://github.com/templetwo/phase-modulated-attention