Schema Constraints, Not Model Capability: A Reproducible Study of Pre-Inference Tool-Calling Failures in llama.cpp
Description
Native tool calling can fail before a large language model receives an opportunity to generate a token. This paper studies such a failure in an OpenAI-compatible llama.cpp deployment of Qwen3-Next-80B-A3B-Instruct Q4_K_M. A reduced request completed a native tool-call cycle, whereas a 12-tool request was rejected with HTTP 400 during sampler initialization. Source and log inspection localized the failure to JSON-Schema-to-GBNF conversion: a valid maxLength: 4000 constraint became a bounded repetition equivalent to char{0,4000}, exceeding the grammar parser's repetition safeguard. We present a schema-scoped compatibility adapter that removes minLength and maxLength only from tools[*].function.parameters, preserves the remaining request and response contract, and leaves the original schema authoritative for pre-execution validation. Seven unit and transport tests passed, and a deployment smoke test completed a multi-step native tool loop with 12 tools after adaptation. The result is a compatibility repair, not an improvement in model reasoning: tool choice and argument correctness remain model-level concerns. The study contributes a failure-localization method, an explicit safety boundary for relaxing generation-time constraints, and a compact reproducible implementation. The underlying parser limitation was independently reported earlier in llama.cpp issue #25923; this work focuses on operational diagnosis and mitigation.
Files
Pyatkov_Schema_Constraints_Tool_Calling_Preprint_v1.pdf
Additional details
Software
- Repository URL
- https://github.com/pyatkovpetr/qwen3nexttoollingadapter