Introducing API Prompt Caching

OpenAI has introduced Prompt Caching to help developers reduce costs and cut latency. For cases where the same context is reused repeatedly, such as code editing tasks or long conversations, recently used input tokens are reused to provide a 50% discount and faster processing speed.

Supported targets and pricing policy are as follows:

• GPT-4o, GPT-4o mini, o1-preview, o1-mini and their fine-tuned versions.
• Cached input tokens are provided 50% cheaper than uncached tokens. (e.g., for GPT-4o, $2.50 $\rightarrow$ $1.25)

Caching is automatically applied to prompts of 1,024 tokens or more, and caches the longest prefix in 128-token increments. Cache usage can be checked via prompt_tokens_details.cached_tokens in the usage field of the API response.

Caches begin to be cleared after 5–10 minutes of inactivity, and are completely removed within a maximum of 1 hour. All caches are not shared between organizations and comply with OpenAI's enterprise privacy policy.