AI Briefing#pruning
The latest AI and developer news about #pruning, with the original source and a short summary.
The latest AI and developer news about #pruning, with the original source and a short summary.
The ToMoE technique has been released, converting dense LLMs into MoE via dynamic structural pruning to reduce active parameters without performance degradation.
LG AI Research presented research on the model compression technique LeSS and LLM structured pruning at an ICLR 2024 workshop.
Ternary Bonsai has released 8B, 4B, and 1.7B models using 1.58-bit ternary weights.
Kanana Nano-2.1B maintains 93% of Kanana Essence's Korean performance despite a 26% size reduction.
Kakao's Kanana Essence, trained on 3T tokens, achieved higher benchmark averages than llama-3.1-8b, while Kanana Nano maximized efficiency through pruning and distillation techniques.