Ubicloud Achieves 100% Tool-Call Accuracy in vLLM via Constrained Decoding

Details

Ubicloud resolved tool-call schema failures in vLLM by activating the constrained decoding server-side flag, raising accuracy from 92% to 100% on the K2VV ToolCall benchmark.
The accuracy drop to 91-92% occurred after migrating from a custom vLLM fork to the upstream vLLM 0.25+ feature, where the default behavior for requests not explicitly enabling constrained decoding changed.
Constrained decoding uses the XGrammar library to mask invalid tokens during sampling, enforcing a strict generation path from tool name to schema-constrained JSON with negligible performance overhead.
vLLM maintainers kept constrained decoding disabled by default to avoid three types of issues: general model performance degradation, grammar translation bugs, and feature interaction bugs with speculative decoding.
Users can now control strictness via the --tool-strict-level flag, allowing deployment-specific configuration for models like Kimi and DeepSeek.

Key point