DeepSeek open-sources DeepInfer inference framework for V4.1-Flash

·2026.10.20 18:08

Key point

The release supports NVIDIA SM100 and Huawei Ascend 950 hardware, featuring optimized kernels and asynchronous scheduling for offline inference.

1 / 3

DeepSeek released DeepInfer, its high-performance inference framework, under the Apache License 2.0.
The framework currently supports offline inference for the DeepSeek-V4.1-Flash model.
Hardware support is limited to NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
Key features include DSpark with confidence-scheduled verification, SWA bounded replay, and batch-invariant inference.
Performance benchmarks were measured on 4 SM100 GPUs for prefill and 64 for decode.
A complete inference service requires additional components not included in this release, such as a KV cache system and load balancing.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.