AI Briefing#mixture-of-experts
The latest AI and developer news about #mixture-of-experts, with the original source and a short summary.
The latest AI and developer news about #mixture-of-experts, with the original source and a short summary.
DeepSeek released previews of V4-Pro and V4-Flash, emphasizing price competitiveness.
Laguna has released the 225B M.1 and 33B XS.2 models along with the ACP runtime.
XiaomiMiMo has released the non-Pro MiMo-V2.5 model with 310B/15B parameter scale.
Compared the performance and weights of 4 abliteration techniques applied to GLM 4.7 Flash.
This is a research paper that directly compares and analyzes the performance and efficiency of MoE and Dense architectures.
DeepSeek cut V4-Pro pricing by 75% and also lowered API cache-hit costs to one-tenth of the previous rate.
Xiaomi has unveiled MiMo-V2.5-Pro, featuring enhanced agentic capabilities and long-horizon task performance.
Explains why the MoE architecture lowers inference costs for DeepSeek's large-scale model.
Expert Upcycling reduces GPU costs by increasing the number of MoE experts midway through training.
Growing a 32-expert MoE to 64 experts mid-training cut GPU hours by about 32%.
Ling-2.6-1T is released as Open Weights, and the same policy applies to the 104B Flash as well.
Qwen3.5-Omni achieves SOTA with 256k context and over 100 million hours of audio-visual data.
BAR trains domain-specific experts separately, merges them into one, and lets you swap out just a few.
MoE models achieve higher inference acceleration when Speculative Decoding techniques are applied.
BAR is an approach where domain-specific experts are each post-trained independently and then merged into an MoE.
The real performance of AI infrastructure is determined not by FLOPS but by cost per token.
A dynamic expert cache in llama.cpp boosted Qwen3.5-122B-A10B token generation speed by 27%.
AI21 Labs' Jamba model addresses the long-context processing limitations of Transformers through a hybrid architecture.
Forge trains, aligns, and evaluates frontier-level AI models using a company's internal knowledge.
Mistral Small 4 has been released, combining reasoning, multimodality, and coding into one model.
This explains the principles and advantages of the MoE (Mixture of Experts) structure, which maximizes the computational efficiency of Transformers.
AI21 has open-sourced Jamba2, splitting it into two models: 3B and Mini.
Since Mamba, hybrid LLMs have been replacing the limits of Transformers.
OpenAI has released gpt-oss, an open-weight model that supports powerful reasoning performance and efficient deployment.