Published August 3, 2026 | Version 1

Schema Constraints, Not Model Capability: A Reproducible Study of Pre-Inference Tool-Calling Failures in llama.cpp

Authors/Creators

  • 1. Independent Researcher

Description

Native tool calling can fail before a large language model receives an opportunity to generate a token. This paper studies such a failure in an OpenAI-compatible llama.cpp deployment of Qwen3-Next-80B-A3B-Instruct Q4_K_M. A reduced request completed a native tool-call cycle, whereas a 12-tool request was rejected with HTTP 400 during sampler initialization. Source and log inspection localized the failure to JSON-Schema-to-GBNF conversion: a valid maxLength: 4000 constraint became a bounded repetition equivalent to char{0,4000}, exceeding the grammar parser's repetition safeguard. We present a schema-scoped compatibility adapter that removes minLength and maxLength only from tools[*].function.parameters, preserves the remaining request and response contract, and leaves the original schema authoritative for pre-execution validation. Seven unit and transport tests passed, and a deployment smoke test completed a multi-step native tool loop with 12 tools after adaptation. The result is a compatibility repair, not an improvement in model reasoning: tool choice and argument correctness remain model-level concerns. The study contributes a failure-localization method, an explicit safety boundary for relaxing generation-time constraints, and a compact reproducible implementation. The underlying parser limitation was independently reported earlier in llama.cpp issue #25923; this work focuses on operational diagnosis and mitigation.

Files

Pyatkov_Schema_Constraints_Tool_Calling_Preprint_v1.pdf

Files (320.5 kB)

Name Size
md5:058aef36fa8d8431bc8424d2fce19322
320.5 kB Preview Download

Additional details