Echo-DSRN-114M: Surprise-Gated Dual-State Recurrent Architecture for Efficient Language Modeling and Classification

Echo-DSRN-114M is a hybrid recurrent architecture combining a GRU fast state, a surprise-gated slow memory state, and bounded sliding-window attention, maintaining O(1) memory with a bounded O(window_size) attention cache fixed at 128 tokens per layer.

The surprise gate uses per-token prediction error to selectively write to long-term memory, outperforming Pythia-160M on zero-shot structured retrieval (SciQ: 0.583 vs 0.519) at 114M parameters and 219 MB fp16.

Weights, training code, LoRA adapters, and live telemetry applications are publicly available under Apache 2.0 in the Echo-DSRN HuggingFace collection.

This is a preliminary working paper reporting results from an undertrained prototype. Findings are provisional and have not been peer-reviewed.