AI BriefingVidai Launches LLM Cost and Routing Tool
Key point
Vidai Community, a Rust-based tool supporting cost tracking, guardrails, and multi-provider routing for LLM calls, has launched.
Details
Vidai Community, which can handle cost management and security for LLM traffic in an integrated way, has launched. This tool is provided as a single Rust binary, and can be integrated immediately by simply changing the base_url in existing OpenAI, Anthropic, and Google GenAI SDKs.
The key features are as follows:
- Multi-provider routing: Bidirectional native translation support that allows routing to OpenAI even when using the Anthropic SDK.
- Cost tracking and budget management: Tracks costs by user, team, and app, with a Hard budget feature that immediately blocks calls when a set budget is exceeded.
- Guardrails and pricing information: Includes a prompt guardrail feature and a free Rate-card service that provides the latest pricing information from major providers.
It also demonstrates excellent efficiency in terms of performance. Its size is only 25MB, and it has verified low overhead of 1.95ms at the median and a performance capable of handling 21,803 RPS on a single node.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Ollama: Fully Ready for Open ModelsForemerge v0.5.0 Released: Rust-Based Open-Source Tool Detects Intent Conflicts Between Parallel AI Coding Agents