AI BriefingDeepSeek V4 KV Cache Usage
Key point
DeepSeek V4's FP16 KV cache usage was recalculated based on a 1M context.
Details
The KV cache figures from Figure 1 of the DeepSeek V4 paper were recalculated to summarize the actual memory usage at a 1M context.
- V3.x(671B): 68.63GiB at 1M tokens
- V4 Flash(284B): 6.08GiB at 1M tokens
- V4 Pro(1600B): 8.71GiB at 1M tokens
The author calculated that with DeepSeek V3.2's MLA KV cache set at 1152 bytes per token, and based on 61 layers, 1M tokens should come out to 68.625GiB. Based on this, the author noted that V4's KV cache reduction isn't simply 9.5x, but rather about 7.879x, and that on a KV-share basis it's closer to a 20x improvement.
The author also projected that if support is added in llama.cpp, running a 1M context could become realistic on systems with sufficient RAM and VRAM.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
DeepSeek Releases DSec: Elastic Sandbox Infrastructure for Large-Scale Agentic Training
Hierarchical Memory Architecture Needed to Resolve AI Inference Bottlenecks