Voodoo Dynamic Quant Released Under MIT License

Developer curvedinf has released the Voodoo Dynamic Quant tool under the MIT License. This technique dynamically selects the optimal quantization level for each tensor in GGUF models, utilizing gradient descent rather than traditional static analysis.

Technical Principles

Voodoo Quant simultaneously runs all quantization levels of the model and uses gradient descent to find the combination that minimizes loss. Specifically, it fixes the candidate quantized weights transformed via the gglm library in llama.cpp and trains only a single scalar gate for each tensor and quantization level. Through Softmax and tau-level annealing, it guides the selection of the single most suitable quantization level for each tensor.

Performance and Features

• Differentiator: It is evaluated as the first publicly available methodology to optimize quantization levels using backpropagation and gradient descent.
• Performance Comparison: While the proprietary Unsloth Dynamic 3.0 may perform better at medium and high quantization levels, Voodoo Quant shows superior performance at aggressive low quantization levels.
• Limitations: It has currently been validated only on small-scale models (such as Qwen3.5), with research on large-scale models ongoing.

Usage

The GitHub repository includes a complete toolset based on the Qwen architecture, designed for rapid adaptation to other model architectures. The author expects this tool to contribute to improving local model performance in low-VRAM environments.