Diffusers Officially Supports Nunchaku 4-bit Quantization

Nunchaku is a 4-bit diffusion inference engine that uses the SVDQuant technique, running both weights and activations at 4-bit (W4A4), which improves speed as well, unlike conventional weight-only quantization. With this Diffusers integration, Nunchaku checkpoints can now be loaded with just from_pretrained(), without a separate inference library.

Performance comparison (RTX 5090, 1024×1024 image generation)

• BF16: ~24 GB VRAM
• Nunchaku NVFP4: ~12 GB VRAM, ~1.7 seconds generation

Core principle of SVDQuant: Activation outliers are absorbed into the weights, the difficult part of the weight matrix is represented with a small 16-bit low-rank branch, and the remaining residual is quantized to 4-bit.

Hardware support

• NVFP4: exclusive to NVIDIA Blackwell GPUs (RTX 50 series, B200)
• INT4: supports previous-generation GPUs

Using the diffuse-compressor toolkit, it's also possible to quantize new architectures yourself and distribute them to the Diffusers repository.