Published September 15, 2025 | Version 1.0
Preprint Open

The Geometry of Forgetting: Toward a Law of Information Decay in Self Modifying Systems

Authors/Creators

  • 1. Hall iNtelligence, LLC

Description

This paper introduces The Geometry of Forgetting, a framework showing that forgetting in self-modifying systems is not a bug but a lawful process. Unanchored knowledge decays with a predictable half-life determined by the spectral properties of the update operator, while conserved anchors guarantee stability.
 
Formally, the framework defines:
1. An update operator mapping model states over time.
2. An anchor score that enforces monotone invariants.
3. A knowledge measure (e.g., mutual information or Fisher trace).
4. A forgetting kernel describing decay outside the anchored subspace.
 
We prove four primitives:
• A recursive data processing inequality with anchor slack.
• An exponential half-life law for unanchored knowledge.
• A Lyapunov anchor pair coupling drift with invariants.
• A no-free-retention inequality linking retention to verification cost.
 
Empirical protocols on continual learning, recursive self-training, reinforcement learning under distribution shift, and symbolic reasoning show how these laws can be tested. Together, this elevates forgetting from an engineering nuisance to a fundamental principle, complementing the Law of Invariant-Preserving Loops and providing measurable bounds on stability, drift, and oversight costs.

Files

The_Geometry_of_Forgetting__Toward_a_Law_of_Information_Decay_in_Self_Modifying_Systems.pdf

Files (649.9 kB)

Additional details

Related works

Is part of
Preprint: 10.5281/zenodo.17103743 (DOI)
Preprint: 10.5281/zenodo.17105319 (DOI)
Preprint: 10.5281/zenodo.17105528 (DOI)

References

  • Kirkpatrick, J., Pascanu, R., Rabinowitz, N., etal.(2017).Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13), 3521–3526.
  • Zenke, F., Poole, B., & Ganguli, S. (2017). Continual learning through synaptic intelligence. International Conference on Machine Learning (ICML).
  • Lopez-Paz, D., & Ranzato, M. (2017). Gradient episodic memory for continual learning. Advances in Neural Information Processing Systems (NeurIPS).
  • McCloskey, M., & Cohen, N. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24, 109–165.
  • Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013). An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211.
  • Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019). Continual lifelong learning with neural networks: A review. Neural Networks, 113, 54–71.
  • French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4), 128–135.
  • Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley.
  • Khalil, H. K. (2002). Nonlinear Systems (3rd ed.). Prentice Hall.
  • LaSalle, J. P. (1960). Some extensions of Liapunov's second method. IRE Transactions on Circuit Theory, 7(4), 520–527.
  • Williams, D. (1991). Probability with Martingales. Cambridge University Press.
  • Hall, J. (2025a). Illusions as Diagnostics, Coherence as Invariant. Unpublished manuscript.
  • Hall, J. (2025b). Beyond Situational Awareness. Unpublished manuscript.
  • Hall, J. (2025c). Intelligence Emerges from Loops, Not FLOPs. Unpublished manuscript.
  • Hall, J. (2025d). The Law of Invariant-Preserving Loops: Toward Robust Emergence in Self Modifying Agents. Unpublished manuscript.