AI Briefing#mixture-of-experts
The latest AI and developer news about #mixture-of-experts, with the original source and a short summary.
The latest AI and developer news about #mixture-of-experts, with the original source and a short summary.
WASTE ran the 2.78 trillion-parameter Kimi K3 on 64GB of memory.
Escha-W2 was released, quantizing the Qwen3.6-35B-A3B model to 2-bit so it can run even on low-spec GPUs.
Moonshot AI has released the weights and technical documentation for Kimi K3, an MoE model with 2.78 trillion parameters.
Moonshot AI has released Kimi K3, an open-weight multimodal model with 2.8T parameters.
GigaChat Audio 10B, an audio-native LLM built on GigaChat 3.1 Lightning, has been released.
Removing some low-weight routed experts in an MoE model was confirmed to have no impact on inference performance.
Comparing two scales of agentic AI through the small model Nanbeige4.2-3B and the large-scale MoE model Laguna S2.1.
AMD has released Instella-MoE-16B-A3B, a new open-source model with a MoE architecture, on Hugging Face.
Poolside's Co-CEO Eiso Kant explains the training infrastructure and strategy behind building Laguna S, a 118B MoE model.
Ant Group's Ling 3.0 Flash model has been added to Vercel AI Gateway and is available for free until August 3.
A new optimizer called SkewAdam reduces optimizer state memory for MoE training from 50.6GB to 1.29GB, enabling 6.78B MoE training on a single 40GB GPU.
InternScience released Agents-A1, a 35B MoE agent model that rivals 1T-class models by scaling agent horizon instead of parameters.
Poolside's open-weight model Laguna S 2.1 has been added to AI Gateway.
MoBA, a technology that applies the MoE principle to attention to maximize the efficiency of long-context processing, has been announced.
Moonshot AI has unveiled Kimi K3, a large-scale open-weight model with 2.8 trillion parameters.
A German research consortium has released Soofi S, a 30B model that surpasses OLMo 3 and Apertus on fully open-source LLM benchmarks.
Covers the evolution of LLMs, foundation models, and technical advances through efficient MoE architecture.
A look at efficient training methodologies and the latest technology trends for foundation models, which serve as the basis of LLMs.
LG AI Research has developed K-EXAONE, a large-scale AI model that maximizes both efficiency and performance.
LG AI Research has developed K-EXAONE, a large-scale AI model that maximizes efficiency and reasoning capability.
LG AI Research has proposed 'Cluster-Aware Upcycling,' a technology that solves the problem of redundancy among experts when converting existing Dense models into MoE.
LG AI Research has unveiled Cluster-Aware Upcycling technology that reduces expert redundancy and maximizes specialization when converting a Dense model into an MoE.
Thinking Machines has unveiled Inkling, a 975B-parameter multimodal open-weight model.
Thinking Machines has unveiled Inkling, a large-scale multimodal MoE model with 1 trillion parameters.