AI Briefing#moe
The latest AI and developer news about #moe, with the original source and a short summary.
The latest AI and developer news about #moe, with the original source and a short summary.
JetBrains has unveiled Mellum2, a 12B Mixture-of-Experts (MoE) model.
JetBrains has unveiled Mellum 2 12B, a small MoE model specialized for coding performance.
NVIDIA has released a quantized version of Alibaba's Qwen3.6-35B model using the NVFP4 data type.
The MiMo V2.5 series maximizes long-context processing and multimodal performance through Hybrid SWA and MoE.
Liquid AI has released an on-device 8B-A1B MoE model trained on 38T tokens with enhanced reasoning capabilities.
StepFun has released Step 3.7 Flash, a 196B-scale MoE-based multimodal model.
MiniMax has previewed its M3 model, which introduces a new sparse attention mechanism to achieve 15.6x faster long-context response speed.
Cohere has released Command A+, an open-source MoE model that ensures data sovereignty for enterprises and the public sector.
This covers the evolution of high-speed GPU-to-GPU communication technology that progressively eliminates CPU intervention to improve distributed training efficiency.
Using OpenAI Triton, a high-performance MoE inference kernel supporting both NVIDIA and AMD has been developed.
A Fused MoE dispatch kernel implemented in Triton delivers higher efficiency than Megablocks along with AMD compatibility.
An MLX port has been implemented that allows Cohere's 218B MoE model, Command A+, to run on Apple Silicon.
DeepSeek is pursuing a grand strategy that goes beyond simply selling models, aiming to build a $10 trillion Chinese AI hardware ecosystem.
Tencent has unveiled the Hy-MT2 series of multilingual translation models supporting 33 languages.
Cohere has released Command A+, an open-weight model based on Apache 2.0.
Cohere has open-sourced Command A+, a 218B-parameter MoE model, under the Apache 2.0 license.
Maximum activation values across 27 open LLM checkpoints varied significantly depending on architecture and training stage.
MTP speculative decoding has been added to llama.cpp, making Qwen3.6 27B up to 2.44x faster.
Ai2 released EMO, showing that performance can be largely preserved using only a subset of a MoE's experts.
A PR has been submitted to llama.cpp adding support for the Sarvam-specific MoE architecture.
llama.cpp has submitted a PR to support the MiMo V2.5 model.
Zyphra has released ZAYA1-8B, a reasoning-specialized MoE model with fewer than 1B active parameters.
APEX, a mixed-precision quantization strategy for MoE models, has expanded to over 30 model types and launched a new ultra-compressed tier.
A reproducible setup was shared for running Qwen 3.6 35B MoE on an RTX 3090 with 262K context.