TOAP: The Token-Optimized Agent Protocol — Reference, Summary, or KV Cache? Measuring How Multi-Agent Systems Should Share Context

TOAP  — Reference, Summary, or KV Cache? Measuring How Multi-Agent Systems Should Share Context

Multi-agent LLM pipelines tend to forward a growing transcript — the source document plus every prior agent's output — into each downstream prompt, so token cost grows with pipeline depth. The obvious fix is to pass a reference to shared content instead of re-sending it. This paper measures that fix honestly instead of advocating for it, and the results are more nuanced than the pitch.

Key Results

Token efficiency (Controlled writer experiment, n=33, Haiku/Sonnet/Opus)

• Reference-minimization (TOAP) and a competent summarizing orchestrator both cut downstream tokens with zero accuracy loss: all 33 samples scored 4/4
• Summarizing baseline saves more tokens than TOAP (∼2.5× vs. ∼1.9× over naive baseline, Table 2) — reference-minimization is not a token win over good prompt engineering

Losslessness under pressure (Pre-registered, n=3 per arm + cross-vendor replication)

• On a task where a faithful 45-word summary necessarily drops 2 of 8 facts, downstream responder scores 8/8 with reference vs. 6/8 with summary, failing exactly on the dropped facts
• Direction replicates on two open, non-Claude models: Qwen2.5-7B (6.0 vs 5.0) and Mistral-7B (5.4 vs 4.4), n=10/arm

Security — capability lattice (Red team, 40 attacks × 9 categories)

• 40/40 attacks blocked, 0 false positives on 14 benign read/analysis requests — 100% block rate at 0% FPR
• Decision is made on (origin, capability), never on text — so phrasing, chaining, or encoding doesn't matter
• Two documented blind spots included by design (within-capability steering; mislabeled origin)

KV-bridge — same-model prefill reuse (7 open models, NVIDIA T4)

• Reusing a transferred prefix cache gives a real, near-lossless prefill speedup: 35/36 cells byte-identical, up to ∼181× at 8k tokens for GQA models (Qwen2.5-0.5B)
• KV cache is 2.5k–43k× larger than the text it replaces — pays off only when agents are co-located or the model uses grouped-query attention, grounding TOAP's gating policy in measurement rather than assumption

Bytes ≠ tokens

• Symbolic "opcodes" save 50% of bytes but only 0–17% of tokens (tiktoken cl100k) — opcode terseness is a minor lever

Multi-hop and coordination

• Exploratory 3-stage pipeline shows 1.65–1.82× downstream token reductions from send-once/refer-many
• TOAP coordination frames are ∼17× smaller in tokens than JSON-RPC equivalents

What this is

A two-plane wire format (binary control plane + tokenizer-aligned semantic plane), a reference-materialization spectrum (KV Bridge → Ctx Ref → Inline) chosen per call by a cost model, a capability-lattice provenance model (Origin × allowed Capability), and a Rust reference implementation (29 unit + integration tests, real-TCP end-to-end). The mechanism is not novel — shared stores, references, and KV reuse all predate this work (blackboard systems, A2A artifacts, ADOL, SEP-1576). The contribution is the honest, reproducible accounting across token, accuracy, security, and compute planes, including a retracted Haiku regression (looked real at n=1, evaporated at n=5) reported as a lesson in why single-call agent measurements are unsafe.

Threats to validity, stated plainly: the token study runs on one model family/tokenizer (Claude); losslessness is one scenario at one compression ratio; the implementation is single-language, single-node, in-memory (durable distributed store is future work).