Intel AutoRound Expands Quantization Support

AutoRound is an LLM/VLM quantization tool based on sign-gradient descent, targeting 2~4bit low-bit inference. It supports CPU(Xeon), Intel GPU(XPU), Gaudi HPU, and CUDA, and integrates with Transformers, vLLM, and SGLang.

Output formats have expanded to include AutoRound, AutoAWQ, AutoGPTQ, GGUF, and llm-compressor, and it reportedly supports 10+ VLMs and 10+ runtime backends. Recently added features include the following.

• block-wise FP8 based on --scheme FP8_BLOCK --iters 0 --disable_opt_rtn
• MTP layer quantization
• SignRoundV2 based on enable_alg_ext and improved INT2
• MXFP4 / NVFP4, mixed-precision AutoScheme, enhanced GGUF

In terms of numbers, a 7B model can be quantized on a single GPU in about 10 minutes, and mixed scheme generation operates with an overhead of about 1.1~1.5x BF16 memory. The default recommendations are auto-round for W4A16, and auto-round-best + enable_alg_ext for W2A16.