Binary Token Memory: A Scalable Compression Framework for Efficient LLM Inference

This work presents the updated and fully binary-based evolution of our previously published Visual Token Memory concept. While the original study explored visual encoding via PNG containers, this paper formalizes an efficient Zstd-compressed binary token memory format designed for scalable LLM applications.

This paper presents a lightweight and efficient binary container format for storing and reusing tokenized representations of LLM conversation histories.
Unlike prior approaches such as image-based Visual Token Memory (VTM v1), the new Binary Token Memory (VTM v2) leverages Zstandard-compressed raw token arrays with minimal headers.
The result is a scalable, GPU-compatible, and persistent memory format that eliminates re-tokenization latency while preserving exact model input structure.
Real-world benchmarks conducted on a modest Intel i5 system demonstrate significant speed and storage improvements over traditional JSON or PNG formats.
The system supports batch encoding/decoding, precise validation, and future extensions for encrypted or multimodal memory.
Aimed at improving production LLM throughput, this method is defensively published to protect prior art and is shared under a CC-BY-NC 4.0 license.