AI BriefingDeepSeek Open-Sources DeepInfer Framework for V4.1 Flash Inference
·2026.10.21 09:00
Key point
The open-source DeepInfer framework supports NVIDIA and Huawei Ascend platforms with a single-cluster throughput of 155 trillion tokens per day.
Details
- DeepSeek open-sources the DeepInfer inference framework and the deepseek-recipe token codec, enabling high-throughput deployment of DeepSeek V4.1 Flash on NVIDIA and Huawei Ascend hardware.
- The system achieves a single-cluster service throughput of 155 trillion tokens per day, with an overall revenue of $38 per GPU per hour at off-peak rates.
- Architecture utilizes disaggregated Prefill (8-GPU nodes) and Decode (64-128 GPU supernodes) with expert-parallel load balancing to optimize end-to-end performance.
- KVCache management employs a 3FS distributed file system for Block KVCache (100 PiB capacity, 3-7 day TTL) and local memory for SWA KVCache, achieving over 98% hit rates for short-term context.
- Performance optimizations include DSpark speculative decoding with variable-length verification (6% throughput gain) and asynchronous scheduling to eliminate GPU idle time during batch processing.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
DeepSeek open-sources DeepInfer inference framework for V4.1-Flash