Prefixwise v0.10.0 released to diagnose and fix LLM prompt cache misses across vLLM, SGLang, and major providers

Details

Prefixwise v0.10.0 is an open-source tool (MIT license) that identifies, fixes, and prevents LLM prompt cache misses for vLLM, SGLang, LMCache, Claude, and OpenAI.
The tool identifies 19 specific causes of cache misses, including tool order changes, timestamp variations, and TTL expiry, agreeing with Anthropic's diagnostics in 32 of 32 cases.
The --fix canonical_tools feature repairs common causes in flight, realizing 99.9% of predicted savings on Claude and 97.5% on vLLM without changing task success rates.
Privacy is maintained by storing only hashes and counts, ensuring no prompt text leaves the local machine.
Integration is available via CLI, Python SDK, LiteLLM callbacks, or as a vLLM middleware, with setup taking approximately 30 seconds per environment.
Live testing on a Colab L4 revealed that reasoning models like Qwen3-4B and DeepSeek-R1 distills re-prefill previous answers due to chat template behavior, accounting for 25-36% and 57% of prompt tokens respectively.
The system adds 3 ms overhead per request and supports data export to JSONL, CSV, Prometheus, and OpenTelemetry.

Key point