AI Briefinglitellm: Manage 100 LLM APIs with a Single OpenAI Format
BerriAI/litellm
About the project
Connect over 100 LLM providers, including OpenAI, Anthropic, Azure, and Bedrock, through a single interface. Eliminate code complexity caused by differing SDKs, authentication methods, and request formats by standardizing on the OpenAI-compatible format.
Integrate directly via the Python SDK or deploy the Proxy Server as a central gateway to control LLM traffic for the entire team. Built-in features such as virtual key issuance, cost tracking, guardrails, load balancing, and logging reduce operational management overhead.
Designed with a Rust core, it achieves a P95 latency of 8ms in environments handling 1,000 requests per second (RPS). Building on stability adopted by large-scale services like Netflix, support has been expanded to include MCP server connections and A2A agent calls.
BerriAI/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Python
This introduction was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report errors, attribution issues, or removal requests via Contact.