CROW: Classifier-Routed Organization of LLM Wikis

LLM wikis have been proposed as an alternative to classical retrieval-augmented generation (RAG): a large language model (LLM) incrementally organizes knowledge into a directory of notes that can be inspected directly and grows with little manual upkeep, at the price of LLM calls whose cost grows with the knowledge base.

Many of these calls are spent on decisions rather than on writing: where a new source belongs, which notes it relates to, which notes answer a question.

We introduce CROW (Classifier-Routed Organization of LLM Wikis), a decision layer for LLM wikis that delegates these routing and consolidation decisions to a general-purpose typed classifier, such as the recently released Jev, and reserves generative calls for the steps that require writing text.
In CROW, ingestion relies on a short sequence of classifier calls and invokes an LLM only to summarize the incoming source, to name new folders and, when an existing note must be updated, to rewrite it, while retrieval uses the classifier to select the relevant notes and leaves to the LLM only the final answer.

CROW does not replace the compilation of entity, concept and synthesis pages that gives LLM wikis much of their value. We specify how each decision maps onto classifier primitives, position the design with respect to memory systems for LLM agents, model routing and candidate-then-verify pipelines, and discuss the effects on cost and latency that we measured in a 525 documents ingestion and retrieval benchmark.