Bayesian Metaplasticity from Synaptic Uncertainty

Catastrophic forgetting remains a challenge for neural networks, especially in life- long learning scenarios. In this study, we introduce MEtaplasticity from Synaptic Uncertainty (MESU), inspired by metaplasticity and Bayesian inference princi- ples. MESU harnesses synaptic uncertainty to retain information over time, with its update rule closely approximating the diagonal Newton’s method for synap- tic updates. Through continual learning experiments on permuted MNIST tasks, we demonstrate MESU’s remarkable capability to maintain learning performance across 100 tasks without the need of explicit task boundaries.