Cursor Releases MoE Kernel for NVL72

Cursor has released Mixture-of-Kittens (MoK), an open-source MoE training megakernel targeting GB300 NVL72 racks.

MoK is designed to fuse the all-to-all communication for dispatch and combine arising from expert parallelism with expert FFN operations into a single CUDA kernel, overlapping communication with computation to hide latency. It also supports deterministic execution, producing bitwise identical outputs for the same input.

Key design directions include:

• Fine-grained communication and computation pipelining leveraging the single NVLink domain characteristic of NVL72
• Ring buffer-based execution structure that reduces CPU intervention and CPU-GPU synchronization
• Optimization of MoE layers using DSV3-style shared experts and hundreds of routing experts
• Tuning push-pull communication methods and communication/computation chunk sizes to match workloads

Cursor reported that applying MoK to Composer training improved end-to-end tokens-per-second throughput by 1.41x on its production stack, and it is currently used for large-scale GPU training involving tens of thousands of GPUs. The implementation is released under the Apache 2.0 license.