AI Briefing#distillation
The latest AI and developer news about #distillation, with the original source and a short summary.
The latest AI and developer news about #distillation, with the original source and a short summary.
A new recipe distills Pythia-1B into a Mamba model, achieving a perplexity of 14.01 compared to the teacher's 13.86.
Kanzhun released the 33B dense model Nanbeige4.3-33B under the Apache 2.0 license.
Confirmed that removing the KL term is essential for preventing performance collapse and improving efficiency when training Qwen3 0.6B with GRPO.
The study found byte-level models required about 6x less text to reach parity with token-based models during initial training.
A study featured on the Apple Research blog proposes TS-DFM to improve the trajectory quality of discrete flow matching, reducing perplexity by 32% compared to a 1,024-step teacher model with 8-step inference and improving speed by 128x.
Apple released PROOF-Gen, which recovers failed tool-calling scenarios to significantly improve model distillation performance.
The QAH technique is released, recovering performance higher than the original model after structural compression and 4-bit quantization.
The Robbyant team released LingBot-World 2.0, a 14B open-source world model capable of real-time interaction at 720p 60fps.
SenseNova has officially launched U1.5-Lite, a single-model inference solution enabled by expert model distillation technology.
Liquid AI has released the LFM2.5 Q4_0 models, which minimize quality degradation through Quantization-Aware Distillation (QAD).
JoyAI has released a 16B model that causally edits live video streams in real time based on natural language instructions.
The AI industry has issued a series of open letters concerning open-weight models and the pace of frontier AI development.
Parallel Decoding Distillation predicts multiple denoising steps at once to improve the speed and diversity of video generation.
Analyzes a turning point in the AI ecosystem centered on the debate over the economics and safety of open-weight models.
A claim has emerged that models, the core competitive advantage of AI companies, can be replicated using the 'distillation' technique, meaning the moat is shallower than thought.
Kimi K3 is the strongest open model available today, but it still trails the closed-model frontier by several months.
Distillation, which began as a model compression technique, has now become a core technique for transferring reasoning ability between models and the center of a copyright controversy.
EdgeRazor is a lightweight LLM framework based on mixed-precision quantization and distillation that can be deployed across diverse hardware.
Hugging Face has released Open-R1, an open-source project to reproduce DeepSeek-R1's training pipeline.
Qwen-Image-Flash demonstrates the importance of optimizing data composition and training pipelines, beyond objective functions, for accelerating image generation models.
Hugging Face released 6 Ettin-based rerankers along with the training recipe.
Needle, an ultra-lightweight AI model with just 26M parameters distilled from Gemini 3.1, has been released.
A NuSLERP merge based on Qwen3.6-27B achieved 0.699 on ARC Challenge.
A single checkpoint switches among 8 presets, delivering up to 10.7x faster performance.