Embedding caching proxy embedcache released

embedcache, a caching proxy designed to reduce GPU resource waste when using self-hosted embedding models such as Ollama, vLLM, and TEI, has been released.

This tool is provided as a single Go binary that runs without any separate Python installation or database, and it offers the following key features.

• Ultra-fast responses: Already-computed text is served instantly from memory in under 1ms.
• Stability and management: It supports a snapshot feature that preserves the cache across restarts, a circuit breaker that prevents service disruption during backend failures, and token budget settings per API key.
• Monitoring: Status can be checked via Prometheus metrics, and an offline analyze command is provided to measure redundant costs by analyzing existing logs.

However, this tool performs caching using an exact-match approach, so it is optimized for content that is completely identical or re-submitted.