Advanced Quantization Algorithm for LLMs

AutoRound is an LLM/VLM quantization toolkit released by Intel, designed using sign-gradient descent to improve accuracy even in the ultra-low-bit range of 2-4 bits. It is connected to SignRoundV1 and SignRoundV2, and emphasizes integration with major inference ecosystems such as Transformers, vLLM, and SGLang.

Recent updates are as follows.

• 2026/03: Supported block-wise FP8 quantization with the FP8_BLOCK combination.
• 2026/03: Added MTP layer quantization.
• 2025/12: Supported mixed-precision reproducibility with enable_alg_ext and the AutoScheme API.
• 2025/11-10: Expanded LLM-Compressor, SGLang, vLLM, and Transformers integration.
• 2025/09-08: Provided MXFP4/NVFP4, improved INT2, and an enhanced GGUF algorithm.

In terms of usage, both CLI and Python API are provided. By default, W4A16 is recommended, with auto-round-best as the accuracy-first recipe and auto-round-light as the speed-first recipe. For W2A16, the auto-round-best combination with enable_alg_ext enabled is recommended.

According to the documentation, a 7B model can be quantized on a single GPU in about 10 minutes, and it also includes support for multi-GPU setups, multiple calibration datasets, 10+ runtime backends, and 10+ VLMs. Output formats have also been expanded to include auto_round, AutoAWQ, AutoGPTQ, GGUF, and LLM-Compressor.