Triton-based Fused MoE Kernel Released

A Fused MoE dispatch kernel implemented using Triton without CUDA has been released.

This kernel recorded 89~131% performance compared to Stanford's Megablocks at inference batch sizes (up to 512 tokens), and achieved high portability, running immediately on AMD MI300X without any code modification.

The key optimization techniques are as follows:

• Gate+Up projection fusion: Designed so that intermediate SwiGLU computation results never leave the registers, cutting global memory traffic by 35%.
• Kernel launch optimization: Significantly reduced the number of kernel launches to minimize overhead.

However, there are limitations: performance falls below Megablocks at batch sizes of 2048 tokens or more, and degradation occurs in environments with 64 or more experts under heavy routing bias.