OmniRoute, a Local Gateway for Multi-LLM Management

OmniRoute is a local-first proxy gateway that unifies SDKs and endpoints from multiple AI providers into one. Users only need to point to a single OpenAI-compatible endpoint, http://localhost:20128/v1, and the tool lets them use various models through it.

Key core features:

• 4-tier automatic fallback: Requests automatically switch in the order of subscription models (Claude Code, etc.) → API keys (DeepSeek, etc.) → low-cost models → free tier, responding to rate limits or outages.
• Various routing strategies: Offers 18 strategies, including priority-based, cost-optimized, headroom-based (remaining quota), and a Fusion strategy that synthesizes answers from multiple models.
• Token compression engine: The RTK and Caveman engines compress repetitive tool outputs (git diff, logs, etc.) to cut token usage by up to 89%.
• Resilience and management: Supports failure isolation via circuit breakers, cooldowns that skip specific keys when their limits are exhausted, and a Quota-Share feature for team-based usage.

It is open-source, built on TypeScript, runs in a Node.js environment, and can be run in various environments including Docker, desktop apps, and PWAs.