DeepSeek API Cuts Costs by 90% with Disk-Based Context Caching

To solve the problem of repetitive input values that occur when using LLM APIs, DeepSeek has introduced Context Caching on Disk technology. This technology caches repeated input content in a distributed disk array, so that when the same content is entered, it is instantly retrieved without recalculation.

When a cache hit occurs, the cost drops to $0.014 per 1 million tokens, reducing costs by up to 90% compared to before. There are also significant performance benefits. When using long prompts at the 128K scale, the first token latency is dramatically reduced from 13 seconds to 500ms.

Users can use the service without any additional code changes, and the cache is automatically applied when the prefix of the input value is the same. The main use cases are as follows.

• Multi-turn conversations: Caching previous conversation context to speed up subsequent responses
• Data analysis: Repetitive queries on the same document or file
• Code and roleplay: Referencing large code repositories or utilizing long character settings

This technology was implemented through DeepSeek V2's MLA (Multi-head Latent Attention) architecture. MLA dramatically reduces the size of the KV cache, enabling efficient storage on low-cost disks. For security, each user's cache is logically isolated, and unused data is automatically deleted.