Ephemeral Context Federation:\\ Giving LLMs Memory Without Becoming Stalkers

This technical note proposes the Ephemeral Context Federation Layer (ECFL): a privacy-first way to give large language models apparent “memory” across conversations, without persistent storage, user profiling, or long-term logging.

Instead of saving embeddings or chat histories in a database, ECFL builds a temporary, in-memory vector index by federating semantically relevant snippets from a user’s prior sessions. The index lives only for the duration of a single answer: it is created on demand, queried once to assemble contextual prompts, and then securely zeroized. No long-lived memory, no cross-session tracking — just recall without retention.

The architecture combines dense retrieval, lexical search, and diversity-aware reranking (MMR), wrapped in a strict token budget and guarded by privacy and safety policies. ECFL can be deployed in server-side RAM, on-device (via WebGPU), or inside trusted execution environments (e.g. SGX/SEV) with remote attestation.

The goal is simple: help LLMs remember what matters in a conversation.