EMA-Gated Temporal Sequence Compressing in Vision Transformers

EMA-Gated Temporal Sequence Compression in Vision Transformers

This release contains the official PyTorch implementation, verification scripts, weights (seperate files), LaTeX source files, and pre-print PDF for the research project: NeuroFlow.

Abstract

We introduce NeuroFlow, a dynamic routing framework for Vision Transformer video inference that exploits temporal redundancy by tracking per-patch semantic surprise via an Exponential Moving Average (EMA) of patch-level embeddings. The central contribution is the Dual-Memory Reconstruction Protocol (Architecture C): a training-free inference engine that combines a Retinal Gate with a Cortical Cache.

Without any fine-tuning or weight modification, Architecture C achieves 71.55% UCF-101 zero-shot top-1 accuracy at 84.0% token sparsity on SigLIP base-patch16-224, retaining 92.4% of dense accuracy. For applications requiring near-2K throughput, Architecture B physically eliminates stationary tokens before the encoder, reducing 1792p SigLIP 2 inference from 678 ms to 11.9 ms—a 55.80× wall-clock speedup at 97.37% embedding fidelity with sparse manifold distillation.

Repository Contents

• Code: Production-ready classes and testing scripts for the NeuroFlow gating architectures (Arch A, B, C, and LLM ablations).

• Paper: Full PDF pre-print and original LaTeX source files.

License Information

Please note the dual-licensing structure of this repository:

• Software/Source Code: Licensed under the Apache License 2.0.

• PDF Document & LaTeX Source: Licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0).