Open-source AI Gateway 'Experiential' Released, Supporting Traffic-Based Routing Optimization

The open-source project Experiential has been released. This tool is an OpenAI-compatible API gateway that integrates hosting, BYOK (Bring Your Own Key), and local models, managing agent workflows.

Key Features and Characteristics

• Unified API and Control: Access various model providers (OpenAI, Anthropic, Gemini, etc.) through a single OpenAI-compatible API, with the ability to set model usage permissions and budget limits per user.
• Traffic-Based Optimization: Collect actual production traffic (OpenTelemetry traces) to generate custom routers or models optimized for quality, speed, and cost. Use the exp build command to build projects based on trace files, and exp optimize to fine-tune open-source models.
• Flexible Deployment: Supports both running the gateway locally (pip install experiential) and using a hosted gateway via platform.experientiallabs.ai.

Use Cases

Developers can upload existing LLM traces or collect real-time traffic to automatically optimize model selection logic for specific use cases. This reduces infrastructure management overhead in AI agent development environments that must balance cost efficiency and response speed.