AI Briefing
KO

Cohere MoE added to vLLM

·2026.04.25 01:04

Key point

A PR supporting Cohere MoE models has been submitted to vLLM.

Details

A PR adding Cohere MoE support to vLLM has been published.

  • It includes a CohereMoeForCausalLM implementation.
  • A SigmoidRenorm routing method has been added to the FusedMoE configuration.
  • An example is provided showing the server launched with vllm serve $MODEL_ID --trust-remote-code and called via the /v1/chat/completions API.
  • It's stated that model weights will be provided soon.
  • Code review pointed out fixes needed such as the scope of RoPE application, residual handling in CohereMoeDecoderLayer, and consistency of the Unspecified type in RoutingMethodType.
inferencemoevllmcohere
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

CohereCohere Releases Megakernel Engine for North Mini Code, Achieving Up to 1.41x Speedup Over vLLMCohereCohere Releases Open-Weight Translation Model 'North Small Translate'… Surpasses Proprietary Models on WMT26RedditOpen-Weight CLM Released as Jev Alternative, Agent Decision-Making 4x FasterKakaoKakao Improves Valid Play Rate by 20% with MoE-Based Personalized Ranking Model Using GDCNHacker NewsStep 5 Preview Released: 600B Parameter MoE Model Highlights Agentic Tasks and Financial Analysis PerformanceRedditMica v0.1 4B Released: Open-Source Decision Model for Agent Loops Trained for Under $30