AI BriefingParoQuant, 4-bit Quantization for Reasoning
Key point
ICLR 2026's ParoQuant has released a 4-bit quantization method along with code and models for reasoning LLMs.
Details
ParoQuant is an ICLR 2026 paper that proposes scaled pairwise rotation to reduce the accumulated error in INT4 quantization of reasoning LLMs.
- Instead of full rotation, it selectively retains pairwise (Givens) rotations that handle only channel pairs.
- It combines per-channel scaling with 8 independent rotation sets to run in parallel on the GPU.
- The entire transformation is packed into a single fused CUDA kernel, with overhead presented as under 10% compared to AWQ.
In the Qwen3-4B 4bit experiment, it recorded MMLU 70.1, GPQA 53.7, AIME24 73.3, AIME25 63.3, and an average of 65.1, outperforming AWQ and EfficientQAT. It claims an average 2.4% accuracy improvement over AWQ on reasoning benchmarks, and other tested models showed a similar trend across the 1.7B~70B range.
It supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX), and also provides an OpenAI-compatible API server and tool-calling agent. The Hugging Face collection includes derivative models from the Qwen3.5, Qwen3.6, Gemma 4, and Llama families.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.