TSCG Empirical Findings: 20,000+ API Call Tool-Schema Compression Benchmark

Comprehensive empirical findings from benchmarking the TSCG (Token-Context Semantic Grammar) tool-schema compiler across 15+ language models and over 20,000 API calls.

This dataset documents systematic empirical discoveries about how production LLMs interact with tool schemas, including four novel findings ("bombshell findings") not predicted by existing literature:

1. Format-Translation Dominance: TSCG's no-operator baseline provides 50-65% token savings across all models, demonstrating that format choice (text vs. JSON) explains the majority of token-cost variance—not compression operators.

2. Per-Version Operator Inversion: Operators that help one model version actively harm the next within the same vendor family. GPT-5.4 inverts GPT-5.2's CFO behavior (-5pp to +15pp). Vendor-pattern hardcoding is empirically broken.

3. Small-Model Sensitive Class: 4B-parameter gemma4-e2b shows identical operator-sensitivity pattern (-7.5pp from CFO) as GPT-4o, refuting paper's vendor-architectural classification. Operator-sensitivity is empirical, not architectural.

4. Combination Fragility: Individual operators classified as helpful or neutral can combine into super-additive negative interactions. GPT-5.5 demonstrates Scenario B: +0/+0/+0 individual operators yield -7.5pp when combined.

Total empirical footprint:
- 20,000+ API calls
- 15+ models tested (4B-32B + frontier)
- 8 operators across 9 sweep conditions
- Cost: ~$50-75 USD

Companion code implementation: ZENODO - TSCG CODE ZIP