TextCLF Releases Calibration-Free 4-bit Quantization Tool

TextCLF has released Quant Factory, an open-source tool on GitHub capable of quantizing Hugging Face models to 4-bit. The tool applies the proprietary TextCLF TQ quantization method, distinguishing itself from existing calibration-based approaches by allowing immediate model conversion without any separate data preparation steps.

Key Features and Characteristics

• Calibration-free: New models can be quantized immediately without separate calibration data, simplifying the workflow.
• Performance: Reported to maintain performance similar to existing calibration-based quantization techniques (e.g., Unsloth UD).
• vLLM Integration: Supports compatibility with the vLLM inference engine and can be served using the --quantization tq flag after installation.

Usage Examples and Deployment Developers can use the provided scripts to quantize models such as Llama 3.1 8B, upload them to Hugging Face, or serve them locally with vLLM. TextCLF has already deployed pre-quantized models (such as Qwen3.8-27B-TQ-4bit) on Hugging Face for immediate testing.