Published March 6, 2026 | Version v1
Working paper Open

Pooling Engram Conditional Memory in Large Language Models using CXL

Authors/Creators

Description

Engram conditional memory has emerged as a promising component for LLMs by decoupling static knowledge lookup from dynamic computation. Since Engram exhibits sparse access patterns and supports prefetching, its massive embedding tables are well-suited for offloading to lower-tier memory. In this paper, we propose using Compute Express Link (CXL) memory pool for Engram storage. Compared to RDMA, CXL provides fine-grained and low-latency access required by minimal and discrete retrieval patterns of Engram. We integrate the CXL-based Engram pool into SGLang, achieving near-DRAM end-to-end performance. This provides a scalable and cost-efficient storage solution for future Engram-integrated LLMs without compromising inference performance.

Files

Pooling Engram Conditional Memory in LLM using CXL.pdf

Files (943.1 kB)

Additional details

Dates

Submitted
2026-03-06