DeepGEMM: LLM-specific CUDA kernel achieving 1550 TFLOPS on H800

DeepSeek released an integrated CUDA kernel library that unifies core operations for large language models into a single codebase. It provides a single interface for complex operations such as FP8, FP4, and BF16-based GEMM, MoE communication overlap, and MQA scoring.

It adopts the DeepJIT approach, which requires no CUDA compilation during installation, reducing deployment overhead. By removing CUTLASS's complex template dependencies and retaining only the core kernels, it offers a clean structure suitable for learning GPU optimization techniques or implementing custom operations.

The Mega MoE feature is designed to perform NVLink communication and tensor core operations concurrently in distributed environments. It fuses EP dispatch, linear transformations, and the SwiGLU activation function into a single mega kernel to minimize bottlenecks.

It supports SM90 and SM100 architectures, achieving up to 1550 TFLOPS on H800. It guarantees processing speeds comparable to or exceeding expert-tuned libraries, improving the efficiency of large-scale inference and training pipelines.