Compressing CoT with Grammar

In the Qwen3.6-27B/35B family, the think block was restricted to a small GBNF grammar, compressing long free-form reasoning into short structured reasoning.

• Running unsloth/Qwen3.6-35B-A3B-GGUF Q4_K_M on a single RTX 6000 with llama-cpp-python on 164 HumanEval+ problems, compared to free-form think, this achieved a 22.4x reduction in thinking tokens along with pass@1 92.1% → 92.7%.
• A simple GOAL/APPROACH/EDGE-style prompt alone still produced many thinking tokens, and it was grammar constraints that actually forced shorter reasoning.
• On the more demanding LiveCodeBench v6 public test set of 50 problems (contest_date >= 2025-01-01), free-form scored 50.0%, while the FSM_PLAN format rose to 64.0%.
• In the same experiment, thinking tokens dropped from 11,553 to 267, and total tokens also decreased from 13,632 to 2,743.
• Free-form execution had many failures such as empty_code, syntax_error, missing_entry_point, and timeout, and grammar reduced these errors by pushing the answer channel into the code stage faster.

The core claim is simple: by leaving the reasoning channel free but strongly constraining only the format of the scratchpad, you can significantly reduce token cost while maintaining performance, or even improve it on some tasks.

However, the author clearly noted the limitations. HumanEval may have contamination issues, these results do not imply a universal compressed-thinking grammar, and this may not hold for long-term planning, SWE-Bench, or multi-file agent tasks.