GCF: A Token-Optimized Wire Format for Structured LLM Interactions

GCF is an AI-native wire format for structured data in LLM systems. 2,500+ evaluations across 11 models and 4 providers. 43 billion+ lossless round-trips across 5 formats. Deployed in 12 production systems.

Comprehension: 100% accuracy on every frontier model on standard workloads. 90.7% on structurally complex data where JSON drops to 54.1% and TOON to 68.8%. No model has ever been trained on GCF.

Token efficiency: 50-92% fewer tokens than JSON (50-69% per call, up to 92% with session deduplication). JSON overhead is 81% structural noise at scale (52.4% repeated field names, 28.6% structural characters); GCF overhead is 0.2%.

Tokenizer safety: GCF is, to our knowledge, the only wire format whose grammar was reverse-engineered from attention-level experimentation and BPE failure mode analysis. GCF's grammar characters were selected from the set empirically verified to have near-zero merge rates across 43 tokenizers from 20 providers. Pipe delimiter: 0.47% merge rate, 24 mergeable words.

JSON: 8.17%, 1,939 mergeable words. TOON tab: 32.91%, 1,238 mergeable words. Controlled experiments across two
architectures (GPT-NeoX, Llama), two scales (410M, 1.3B), and three domains (structured data, code, molecular notation) prove that preventing delimiter merges produces 3-738x better structured data comprehension, with zero natural language cost. Every attention head in a standard BPE model is structurally constrained by delimiter merging; the model has 4x more structural attention capacity than the tokenizer allows it to use. See companion paper: "Tokenizer-Attention Coupling" (DOI: 10.5281/zenodo.20925910).

Six implementations (Go, TypeScript, Python, Rust, Swift, Kotlin), 157 conformance fixtures, zero dependencies. 

Specification v3.2 Stable. Lossless across 17 serialization formats.