One Bit at Every Scale: A Cross- Disciplinary Analysis of Uncertainty Transitions

Many physical, biological, and computational systems can be described as transitioning from a probability distribution to a realized state. Using Shannon’s definition of entropy, this paper quantifies the informational content of such transitions across four domains: quantum physics, cosmology, large language models (LLMs), and neural systems. For each domain, we identify a representative elementary transition, derive or extract the relevant probability distribution, and compute its entropy in bits. Across spin measurements, tunneling measurement outcomes, inflationary modes, black-hole area elements, next-token distributions in transformer models, EEG microstates, MEG phase states, and decision-related neural activity, the entropy of an elementary transition consistently falls within a narrow range of approximately one to two bits.

The comparison distinguishes between transitions whose distributions are directly measurable (quantum experiments, cosmological spectra, electrophysiological data) and transitions whose distributions are effective or inferred (internal gating events in LLMs). The resulting numerical regularity therefore reflects similarities in probabilistic structure rather than shared physical mechanisms.

While these systems differ fundamentally in scale and substrate, the convergence in informational scale is empirically robust across independently motivated definitions. Whether this pattern reflects deeper physical constraints or methodological commonalities remains an open question warranting further investigation.

This descriptive informational comparison provides a quantitative reference for cross-domain analysis. Future work may explore whether the informational scale of state transitions supports methodological transfer between fields or reveals deeper structural constraints on how systems reduce uncertainty.