MiMo v2.5 Inference Optimization: Pushing Hybrid SWA Efficiency to the Extreme

The MiMo-V2.5 model series is designed to combine Hybrid Sliding Window Attention (Hybrid SWA), Sparse MoE, and a multimodal encoder to secure both long-context processing capability and efficiency at the same time.

Hybrid SWA uses Full Attention for a subset of the total layers (10 layers for MiMo-V2.5-Pro), while the remaining 60 layers use Sliding Window Attention (SWA). This achieves the following performance improvements.

• Compute reduction: Compared to Full Attention, computational complexity is lowered to about 1/7 the level, significantly reducing the cost of the Prefill stage.
• KVCache storage optimization: Since SWA layers only retain KV within the window, memory usage is reduced to about 1/7, shortening latency in the Decode stage.

This article covers the engineering practices for implementing these theoretical benefits in an actual production environment. The key optimization elements are as follows.

• KVCache management: A tiered caching system and application of a Prefix Cache Tree that accounts for SWA
• Scheduling: Optimization of the Prefill/Decode execution pipeline and synchronization of distributed cache state
• MoE and multimodal optimization: Load balancing for distributed scheduling and resolving throughput bottlenecks in the multimodal encoder