Breaking Through DeepSeek-V4-Pro Serving Limits

LMSys detailed the engineering optimization process for efficiently serving the DeepSeek-V4-Pro model in an H20 GPU environment. The core approach involves analyzing hardware constraints to design customized serving profiles for the Prefill and Decode stages.

First, capacity efficiency was maximized by reducing weight size through Humming MXFP4AFP8 quantization and expanding KV cache capacity by applying Online C128. This enabled the configuration of serving profiles suitable for various scenarios.

In the Prefill stage, the MoE-TP method was adopted to balance communication and computation, with tuning performed to align with the Humming routing format. In the Decode stage, DSpark was expanded across pipeline stages to ensure low latency, while the DP32-EP32 configuration eliminated high-concurrency bottlenecks to increase throughput.

Final evaluation results confirmed overall system performance improvements through these optimizations, demonstrating profiling techniques that effectively manage the trade-off between context length and throughput.