AI Briefing#model-compression
The latest AI and developer news about #model-compression, with the original source and a short summary.
The latest AI and developer news about #model-compression, with the original source and a short summary.
The model compresses MiniMax-H3 to 13.7B parameters for 4-8 step generation, running on consumer RTX 5090 GPUs in 92 seconds for 5.2-second clips.
Fine-tuned on 2B tokens of synthetic data, the model predicts screen deltas from keypresses, outperforming prompted baselines on shell and file readback tasks.
The model retains 98.2% of FP16 intelligence while reducing size to ~5.9 GB for llama.cpp.
The method achieves 2.8× compression while staying within 1.9% relative WER of the teacher model on five of six pairs.
Researchers at the University of Waterloo have released PAW, an open-source framework that compiles English text descriptions into small Neural Programs executable locally.
Cerebras has started serving the Qwen 3.8 27B model on its public endpoint at a speed of approximately 1,500 tokens per second.
NVIDIA has released Model Optimizer, which integrates six optimization techniques including quantization and pruning to support checkpoint generation for inference engines such as vLLM and TensorRT-LLM.
The IDEA Prune pipeline, which integrates pre-training of expanded models with pruning, improved LLM compression performance.
The QAH technique is released, recovering performance higher than the original model after structural compression and 4-bit quantization.
The ToMoE technique has been released, converting dense LLMs into MoE via dynamic structural pruning to reduce active parameters without performance degradation.
CKA-QAD improves inference and coding performance by preserving internal representations during NVFP4 quantization-aware distillation.
EdgeRazor improves preservation of teacher LLM performance even at 1.88-bit precision.
RaBiT is proposed as a binarization training technique that leverages residuals to improve accuracy while maximizing the efficiency of LLMs.
It proposes SLQ, a statistically lossless quantization technique that increases inference speed while minimizing performance degradation in LLMs.
At ICLR 2024, LG AI Research introduced LeSS, a new model compression technique that combines filter pruning and low-rank decomposition.
LG AI Research presented research on the model compression technique LeSS and LLM structured pruning at an ICLR 2024 workshop.
PrismML unveils technology that compresses the Qwen-3.6-27B model to under 4GB to run on iPhone.
ReFreeKV, a threshold-free KV Cache compression technique that automatically adjusts the compression budget based on input data, is proposed.
Distillation, which began as a model compression technique, has now become a core technique for transferring reasoning ability between models and the center of a copyright controversy.
The Sana 1.6B text-to-image model was quantized to 1.58-bit ternary, reducing its size by 8.6x.
Moebius is a lightweight inpainting framework that, through LλMI blocks and an adaptive distillation strategy, reduces parameter count to under 2% while delivering performance comparable to large-scale models.
They unveiled **InstinctRazor**, a technology optimized to run MoE models on edge devices.
Google has released optimized models applying QAT technology to run Gemma 4 efficiently on mobile and edge devices.
NVIDIA has unveiled Star Elastic, a technology that can extract models of various sizes from a single checkpoint.