AI BriefingGemma 4 QAT MTP Heads Released and Bug Fixed
Key point
MTP heads for Gemma 4 QAT models have been released, along with news of a fix for a llama.cpp parallel execution bug.
Details
MTP (Multi-Token Prediction) assistant heads matching Google's official Gemma 4 QAT Q4_0 models have been released on HuggingFace.
These heads address the performance degradation that occurs during Speculative Decoding due to the distribution gap between existing non-QAT heads and QAT models. Test results showed that using QAT-specific heads significantly improved the Acceptance Rate for each model as follows:
- 12B QAT: 71.3% → 78.4% (+7%p)
- 26B-A4B QAT: 56.9% → 91.8% (+35%p)
- 31B QAT: 42.5% → 60.4% (+18%p)
Additionally, a crash bug occurring with the PARALLEL=2 setting in the Atomic fork and llama.cpp was fixed, and the bug was submitted to the official llama.cpp repository. The released heads use the gemma4_assistant architecture and are immediately available in the Atomic TurboQuant fork.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Optimizing Gemma 4 Serving with vLLM on Amazon EKS [Part 2: Throughput and Latency SLOs on a Single GPU]
The Efficient Frontier of LLM Inference (6-minute read)Unsloth Dynamic 3.0 GGUF Released