AI BriefingIntel Unveils AutoRound, a High-Performance Quantization Tool
Key point
Intel has released AutoRound, a tool for LLMs/VLMs that maintains high accuracy even at low-bit quantization.
Details
AutoRound is a weight-only post-training quantization (PTQ) methodology developed by Intel. By using signed gradient descent to jointly optimize weight rounding and clipping ranges, it minimizes accuracy loss even at low-bit quantization in the INT2–INT8 range.
In particular, at 2-bit (INT2), it shows up to 2.1x higher relative accuracy compared to existing baselines, and it also maintains competitive performance at 4-bit.
Key Features and Advantages:
- Fast Speed and Efficiency: It is lightweight and fast enough to quantize a 72B-scale model on an A100 GPU in about 37 minutes, and can achieve high accuracy with just 200 tuning steps and a small dataset of 128 samples.
- Broad Model Compatibility: It supports major LLMs such as Qwen, LLaMA, and DeepSeek, as well as 10+ VLMs including Mistral and Gemma3.
- Diverse Environment and Format Support: It supports CPU, Intel GPU, and CUDA devices, and can export models in formats such as GPTQ, AWQ, and GGUF.
- Flexible Configuration: It supports mixed-bit tuning and lm-head quantization, enabling optimization tailored to user needs.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.