AI BriefingGemma vs Qwen KV Cache Comparison
Key point
Gemma was sensitive to KV cache quantization, while Qwen held up relatively well.
Details
Comparing q8_0 KV cache and q4_0 KV cache against the f16 baseline revealed a major difference in sensitivity by model.
- Gemma 4 showed clear quality degradation even at q8_0, and broke down further at q4_0.
- Qwen 3.6 maintained under KL 0.04 at q8_0, and even q4_0 was measured as usable for practical purposes except in some ranges.
- In particular, Gemma 4 26B A4B (MoE) was the most sensitive in this test, with the q8_0 cache worsening to KL 0.377 and q4_0 to KL 1.088.
- In contrast, Qwen 3.6 35B A3B (MoE) did not show the same level of amplification despite also being an MoE architecture.
By category, Gemma degraded fairly evenly across the board, while Qwen's losses were concentrated in long documents and tool calls.
The measurements were performed under BF16 GGUF, on the same machine, with the same dataset conditions, changing only f16 cache / q8_0 / q4_0, using about 250,000 tokens, 6 categories, and token-by-token comparison via KL divergence.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Apple Releases LensVLM-9B for Processing Compressed Text Images
Qwen Releases 'RecreationWorld' for Training AI Agents to Rebuild AppsQwen Intelligence Launches Three Mobile AI Agents and Open Benchmark Suite