There is a newer version of the record available.

Published May 28, 2025 | Version v2

Bridging Shannon and Turing: A Formal Entropy–Complexity Correspondence over Symbolic Structures

Description

We establish a formal correspondence between Shannon entropy and Kolmogorov complexity
in symbolic domains governed by constrained rewrite systems. By defining a class of canonical
representations for symbolic structures, we construct a bridge between statistical compressibility
and algorithmic compressibility. We prove that for any computable distribution over irreducible
forms, the expected Kolmogorov complexity deviates from Shannon entropy by at most a con-
stant plus a structural degeneracy term. This yields tight two-sided bounds unifying probabilistic
and algorithmic theories of information. Applications include lossy compression lower bounds,
grammar-constrained learning, and complexity-preserving communication protocols.

Files

Bridging_Shannon_and_Turing__A_Formal_Entropy__Complexity_Correspondence_over_Symbolic_Structures_.pdf

Additional details

Dates

Created
2025-05-18
Updated
2025-05-28