EdgeRazor, 1.88-bit LLM

EdgeRazor is a lightweight framework that uses entropy-based distillation to transfer the teacher model's logit probability distribution into the student model's low-bit, mixed-precision hidden layer features.

Rather than preserving the teacher model's parameter structure as-is, it trains the student model to retain more of the teacher's capabilities even at extremely low-bit precision. The authors validated performance at a precision level of 1.88 bits/parameter.

• It requires more computation than existing quantization techniques, but the authors explain it is lighter than typical quantization-aware training (QAT).
• They claim it does not introduce a separate internal representation, so it can be used without modifying existing inference implementations.
• The publicly released models include MobileLLM, Qwen3-0.6B, Qwen3-1.7B, and Qwen2.5-Omni-7B, and the current example models are relatively small due to training costs.

The paper and code have been released on arXiv and GitHub, and the trained models are provided as a Hugging Face collection.