AI Briefing#mixture-of-experts
The latest AI and developer news about #mixture-of-experts, with the original source and a short summary.
The latest AI and developer news about #mixture-of-experts, with the original source and a short summary.
Qwen-MT boosts performance with 92-language translation, customization, and low-cost, high-speed processing.
Kakao has open-sourced 'Kanana-1.5-15.7B-A3B', a model utilizing an MoE structure to achieve performance comparable to Dense 8B models with only 3B active parameters.
The 480B MoE Qwen3-Coder aims for SOTA in coding, browser use, and tool use.
Applying load balancing at the global-batch level instead of the micro-batch level significantly improves MoE model performance and expert specialization.
xAI has introduced a new image generation model, Aurora, into Grok and launched it on the 𝕏 platform.
AI21 has released Jamba, a Mamba-based model supporting a 256K context window.
Qwen1.5-MoE-A2.7B achieves performance on par with 7B models while reducing activated parameters to 1/3.
xAI has released the weights and architecture of **Grok-1**, a MoE model with 314 billion parameters, under the Apache 2.0 license.
SegMoE, a Mixture-of-Experts (MoE) framework for diffusion models, has been released.
This explains in detail how the MoE architecture works, how it is trained, and the trade-offs during inference.
Mistral AI's new MoE model Mixtral 8x7b has been released, and it outperforms GPT-3.5.