Beyond RAG: LLM Wikis as Living Semantic Memory: Patterns, Empirical Findings, and Open Problems

Large language models have reached the point where a single context window can hold hundreds of documents, yet the dominant retrieval-augmented generation (RAG) paradigm still reconstructs knowledge from scratch on every query. Andrej Karpathy's April 2026 proposal for an LLM Wiki—a persistent, structured Markdown knowledge base that an LLM incrementally compiles and maintains—offers a different design point: knowledge is compiled once and kept current, rather than re-derived on every query. This position paper reports our experience exploring that design point across a series of empirical studies, open-source knowledge base deployments, and domain applications conducted in the period following the original proposal. We make three main arguments. First, the LLM Wiki pattern is best understood not as a replacement for RAG but as a compiled semantic memory for an LLM agent. Second, the most practically significant extension we identified is treating the wiki as a living constraint oracle (LCO) for safe offline reinforcement learning. Third, integrating formal symbolic reasoning—Prolog, ProbLog, Answer Set Programming—as a first-class module addresses the fundamental limitation of narrative-only knowledge compilation. We also report that the wiki pattern failed to outperform RAG in two tested domains, and that in its current form the pattern is bounded by the scale of a personal knowledge base.