AI Briefing#prompt-caching
The latest AI and developer news about #prompt-caching, with the original source and a short summary.
The latest AI and developer news about #prompt-caching, with the original source and a short summary.
The MIT-licensed tool analyzes KV-cache events from vLLM and SGLang to pinpoint why requests miss the prompt cache, achieving 32/32 agreement with Anthropic's diagnostics.
Prefixwise v0.10.0 identifies and fixes LLM prompt cache misses, realizing 99.9% of predicted savings on Claude and 97.5% on vLLM.
AI Gateway and AI Studio provide integrated management of LLM and Agent calls while controlling costs and security.
Model switching and MCP server additions invalidate the cache, while file edits and subagents do not, with a default TTL of one hour for subscriptions.
Factory releases Droid 1.0, featuring Programmatic Tool Calling (PTC) and a 37% reduction in first-request input tokens.
The update improves prompt-cache reuse and incremental context handling, requiring no user action.
The updates fix prompt caching bugs, add /usage tracking, and prevent context bloat in long sessions and multi-agent runs.
New features include a Decisions API for efficient routing, zero-cost voice design, and GPT-Live-1.1 for faster real-time responses.
Danggeun reduced costs to one-fifth by adding explicit breakpoints after costs increased 3.2x due to automatic cache rule changes following the transition to GPT-5.6.
This article summarizes Uber's case study on reducing AI agent costs and strategies for measuring and optimizing token costs using Amazon Bedrock.
Optimizing prompt caching and switching to Sonnet 5.5 reduced median review time from 4m 09s to 1m 32s while maintaining finding quality.
Cache misses caused by harness modifications like timestamp changes or tool list updates increase energy consumption by up to 2.5x compared to fully cached requests.
The update allows tools to load only when a skill is read and enables mid-thread skill reloading without starting a new session.
OpenAI has released a comprehensive model guide for the GPT-6 family, detailing strategies for managing context costs, selecting reasoning levels, and handling long-running workflows across GPT-6 Astra, GPT-6.1 Sol, and GPT-6 Luna.
OpenAI has released new features and guides, including explicit cache breakpoints, prewarming, and a diagnostic dashboard, to improve prompt caching efficiency for GPT-6 models.
An optimization method has been released to reduce context rewrite costs after long tool calls by changing the default 5-minute cache TTL for Claude Code sub-agents to 1 hour.
It has been reported that Claude Code fails to reuse the prompt cache for follow-up requests after a sub-agent terminates, resulting in the rewriting of nearly the entire context and incurring charges.
This article presents methods for managing Claude Code prompt cache TTL and controlling organizational costs in cloud environments such as Amazon Bedrock.
The guide recommends a single-agent architecture with skills over subagents to reduce latency and cost, supported by a GitHub reference implementation.
Since Claude Code charges proportionally to context size, costs should be optimized by improving prompt cache hit rates and appropriately using session management commands such as /clear and /compact.
This article presents optimization strategies to reduce token costs and improve efficiency when using Claude Code.
The guide outlines how IT admins can use access gating, model entitlements, and hard spend caps to manage organizational spending on Claude.
As Inference APIs bind the core state of sessions to the provider, data ownership and portability are being weakened.
Prompt Caching reduces the cost of agents, but it reacts very sensitively to configuration changes.