AI Briefing#moe
The latest AI and developer news about #moe, with the original source and a short summary.
The latest AI and developer news about #moe, with the original source and a short summary.
GSPO reduces GRPO's instability through sequence-level optimization, enabling RL scaling.
Kakao has open-sourced 'Kanana-1.5-15.7B-A3B', a model utilizing an MoE structure to achieve performance comparable to Dense 8B models with only 3B active parameters.
Qwen3 is an open-weight model with a hybrid thinking mode and support for 119 languages.
Meta's next-generation MoE models, Llama 4 Maverick and Scout, have been released on Hugging Face.
Alibaba unveiled **Qwen2.5-Max**, a large-scale MoE model trained on over 20 trillion tokens.
Applying load balancing at the global-batch level instead of the micro-batch level significantly improves MoE model performance and expert specialization.
Qwen1.5-MoE-A2.7B achieves performance on par with 7B models while reducing activated parameters to 1/3.
This explains in detail how the MoE architecture works, how it is trained, and the trade-offs during inference.
Mistral AI's new MoE model Mixtral 8x7b has been released, and it outperforms GPT-3.5.
Naver Cloud has joined NVIDIA's Nemotron Coalition, beginning collaboration to advance HyperCLOVA X and secure sovereign AI.