TOPO-JEPA: A Topological Joint Embedding Predictive Architecture for Continual World Models

Core Concept

TOPO-JEPA is a novel architecture designed to enable continual learning in world models by integrating two complementary approaches:

• TOPO (Topological AI): Provides a mathematical guarantee of memory preservation to solve catastrophic forgetting, using $O(1)$ memory overhead (67.5 KB).

• JEPA (Joint Embedding Predictive Architecture): Provides self-supervised, task-agnostic representation learning for world model prediction, though it does not address catastrophic forgetting on its own.

• Full code 

Key Performance and Findings

• Zero Catastrophic Forgetting: Through the mathematical guarantees provided by TOPO, the model avoids the traditional degradation of performance when learning new tasks.

• Negative Forgetting: Experimental results showed -0.75% combined forgetting, meaning the model actually improved on earlier tasks after learning new ones.

• High Accuracy: The architecture achieved 89.0% accuracy on the "World vs Sci/Tech" cross-domain task, which was the hardest task in the protocol.

• Efficiency: The $O(1)$ memory cost of 67.5 KB is 34,000 times less than the HOPE (Nested Learning) architecture.

Distinguishing Roles

Technical Foundation

• Prime-Anchored Embeddings: Six embedding rows at prime indices (2, 3, 5, 7, 11, 13) are frozen after the first task.

• Arithmetic Spectral Theory: The safety constant $\Lambda = 0.9785142874$ provides the theoretical guarantee for anchor stability.

• Integration: The Topological Governor only requires three additions to standard fine-tuning loops: taking a snapshot after the first task, zeroing anchor gradients and enforcing anchors during subsequent training, and verifying integrity.

Availability

The implementation and the certified model are available on GitHub and the Hugging Face Hub.