Attention Sinks at Long Context: One Log-N Law for Retrieval and Abstention

Softmax attention cannot abstain on its own: its weights sum to one, so a head with nothing relevant to read must still attend somewhere. Trained models improvise such a place, the attention sink. Practice now builds one in: a null, an option that absorbs attention and delivers nothing, such as a learned sink logit, a register with zero value or an output gate. We show that these nulls are one mechanism, a gate on an ordinary softmax, and ask what happens to them as the context grows. The answer is one dilution law for two abilities. To retrieve, a matching key must outweigh the N other keys; to abstain when nothing matches, the null must. With bounded scores both leads must grow like log N, and because a query with a match and one without share the same null, no rule that sums over the keys keeps both at long context: a fixed null loses abstention, and a null that grows with the length loses retrieval. Tempering, which scales every score, the null included, by log N / log N_train, provably keeps both under margin conditions, and a head that decides by its largest score escapes the law. Trained heads in a controlled pilot follow the law closely, and tempering keeps both sides there, though not in every harder setting. In 125M-parameter language models, every rule with a null removed the first-token sink and most massive activations; past the training length, keeping positions in range mattered more than the null rule, and adding tempering to YaRN, which already includes a temperature, hurt. In small vision transformers trained at one resolution, tempering at evaluation raised accuracy at higher resolutions. A prespecified test on gpt-oss was inconclusive.

This is a working draft, not a final version. It is unpolished in places, incomplete in others, and keeps exploratory material a polished paper would leave out, because that material helps decide what comes next and shows the programme's big picture more fully. Please read it with that in mind: the work will be refined further, and later versions will appear under this record. It belongs to a collection released together on Zenodo: the entry paper The Transformer Abides, the monograph A Spectral Theory of Token Communication, and the papers Attention Sinks at Long Context, The Transformer as a Constrained Equilibrium and Beyond Transformers. This version is the text; the records, the code and the formal proofs follow in a later version.