There is a newer version of the record available.

Published December 2025 | Version v4
Preprint Open

Semantic Grounding and the Preservation of Information in Recursive Systems

Authors/Creators

Description

This article presents an information-theoretic framework explaining model collapse in self-referential learning systems. Four premises establish that: (I) endogenous semantic drift is inevitable in closed loops; (II) long-term stability requires a viability condition where corrective bandwidth exceeds the error rate (Ceff(t)E(t)); (III) systems violating this condition undergo informational autophagy; and (IV) this failure mode exhibits a distinct, falsifiable temporal signature.

Specifically, the framework predicts that during recursive training, out-of-distribution accuracy will degrade before validation perplexity rises. This temporal lag distinguishes semantic divergence (loss of grounding) from capacity-driven collapse (general degradation). By reframing synthetic contamination from a binary risk to a quantitative rate problem, the theory demonstrates that scaling with synthetic data is viable only when paired with commensurate verification infrastructure.

Files

Semantic Grounding and the Preservation of Information in Recursive Systems (v4).pdf

Files (684.7 kB)

Additional details

Dates

Created
2025-12
Initial draft

References

  • Alemohammad, S., Casco-Rodriguez, J., Luzi, L., Humayun, A. I., Babaei, H., LeJeune, D., Siahkoohi, A., & Baraniuk, R. G. (2023). Self-Consuming Generative Models Go MAD. arXiv preprint arXiv:2307.01850. https://arxiv.org/abs/2307.01850
  • Apple Machine Learning Research. (2025). The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models. https://machinelearning.apple.com/research/illusion-of-thinking
  • Arulandu, A. C., et al. (2025). Position: Model Collapse Does Not Mean What You Think. arXiv preprint arXiv:2503.03150.
  • Dohmatob, E., Feng, Y., Yang, P., Charton, F., & Kempe, J. (2024). A Tale of Tails: Model Collapse as a Change of Scaling Laws. Proceedings of the 41st International Conference on Machine Learning (ICML). https://proceedings.mlr.press/v235/dohmatob24b.html
  • Feng, J., et al. (2025). Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification. ICML/ICLR-linked work. arXiv:2406.07515.
  • Gebendorfer, J. (2025). Semantic Physics: The Fundamental Framework (Field, Fugue, Fold and the Bi-Modal Architecture of Meaning). ResearchGate. https://www.researchgate.net/publication/397097076_Semantic_Physics_The_Fundamental_Framework_Field_Fugue_Fold_and_the_Bi-Modal_Architecture_of_Meaning
  • Gerstgrasser, M., Schaeffer, R., Dey, A., Rafailov, R., Sleight, H., Hughes, J., Korbak, T., Agrawal, R., Pai, D., Gromov, A., Roberts, D. A., Yang, D., Donoho, D. L., & Koyejo, S. (2024). Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. arXiv preprint arXiv:2404.01413. https://arxiv.org/abs/2404.01413
  • Lee, T. (2025). Synthetic Data and the Future of AI. Cornell Law Review.
  • Ma, L., et al. (2025). A Semantic Energy-Aware Ontological Framework for Adaptive Task Planning and Allocation in Intelligent Mobile Systems. Electronics, 14(18), 3647. https://www.mdpi.com/2079-9292/14/18/3647
  • Satharasi, T., & Iyengar, S. S. (2025). Future of AI Models: A Computational Perspective on Model Collapse. arXiv preprint arXiv:2511.05535.
  • Shi, L., et al. (2025). A Closer Look at Model Collapse: From a Generalization-to-Memorization Transition. NeurIPS 2025. arXiv:2509.16499.
  • Straňák, P. (2025). Lossy Loops: Shannon's DPI and Information Decay in Generative Model Training. Preprint.
  • Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Anderson, R., & Papernot, N. (2023). The Curse of Recursion: Training on Generated Data Makes Models Forget. arXiv preprint arXiv:2305.17493. https://arxiv.org/abs/2305.17493