AI Briefing#moe
The latest AI and developer news about #moe, with the original source and a short summary.
The latest AI and developer news about #moe, with the original source and a short summary.
Runs on laptops with 2.45B active parameters, outperforming Qwen3.6 in tool-calling performance
The model features 42B active parameters per token, a 1M-token context window, and supports text, image, audio, and video inputs.
The new 35B-A3B MoE model utilizes Lookahead Expert Scheduling to enable local vision-language processing on smartphones.
The 74B parameter MoE model is packed into 24GB using 2-bit quantization, with a 4GB variant releasing Oct. 23.
The open-weight model achieves 184 tokens/s on an Apple M5 Max and scores 48.09 on LiveClaw, outperforming larger models in agentic tasks.
The experimental C-based Janas-LLM achieves 24 tokens/s generation for the 48.4GB Qwen3-Next-80B-A3B on a 32GB RAM laptop, significantly outperforming llama.cpp.
The new open-weight model features only 2B active parameters, achieving up to 1.7x faster inference than comparable MoEs on-device while matching larger models on key agentic benchmarks.
The new model achieves 13+ tokens/s decoding speed on iPhone 17 Pro Max using Lookahead Expert Scheduling.
The new framework allows a single model to span 1, 2, 3, or 4 billion parameters, outperforming dense counterparts by 2-5% on knowledge-intensive benchmarks.
Scheduled for release in October under the Apache 2.0 license, this sparse MoE architecture emphasizes inference compute efficiency.
Routing transplant experiments on OLMoE and DeepSeek models show that forcing morality-associated expert paths significantly alters malicious code detection accuracy, independent of the model's readable workspace.
Reflection AI has introduced Beam, a 501B-parameter open-weight MoE model trained on 10.5K NVIDIA GB300 GPUs, claiming high inference efficiency compared to larger models like GLM 5.2 and Kimi K3.
Achieved an AA Index score of 24.1 with an MoE architecture that activates only 3B of its total 35B parameters
Aleph Alpha has released Kolibri, an open-weight English-German Mixture-of-Experts model with 78.1 billion parameters and support for 1 million token contexts, available under the Apache 2.0 license.
AlephAlpha has released Kolibri, an open-weight Mixture-of-Experts model with 78.1B total parameters and 3.46B active parameters, designed for sovereign and regulated domains with a focus on German language performance.
Ai2 releases Olmo-core 3, an open-source training framework designed for large MoE models, achieving 2.7x throughput improvement in preliminary tests and benchmarked at over 1 trillion parameters.
The model features 320B total parameters with only 15B activated per token, optimized for reasoning and multi-step tool use.
The open-source vllm-xtu-moe project achieves up to ~20× faster prefill than CPU-only paths for medium contexts by streaming expert weights from CPU to GPU.
The new Cyber-OSINT model features 26B total parameters with 4B active, a 262K context window, and is trained on 6,500 OSINT/CTI instructions.
StepFun has released Step 5 Preview, offering intelligence on par with Kimi K3 at one-third the cost, with weights scheduled to open on October 15.
Kakao introduced a personalized ranking model combining GDCN and MoE, improving the valid play rate by 20%.
The MiMo-V2.6-Pro released by Xiaomi recorded higher intelligence scores compared to similar-sized open-weight models and features competitive pricing, though its output speed was found to be slower than average.
A scheduling technique has been released that manages the four major memory peaks occurring during MoE training with long contexts or large batch sizes within a fixed GPU working set.
Xiaomi has launched three MiMo V2.6 models supporting coding and multimodal capabilities on AI Gateway.