Weave Router Released: Local Embedding-Based LLM Routing

Coding agents generate multiple API requests, such as file reads and command executions, for a single user input, leading to rapidly escalating costs. Weave Router is a routing proxy that automatically selects the appropriate model for each request.

Local Embedding-Based Decision Structure

Instead of separate LLM calls or rule-based configurations, Weave Router uses a small embedding model running within the router process. It converts request text into embedding vectors, compares them against pre-computed cluster centroids, and selects a model from the ranking list of the corresponding cluster. This approach follows the cluster scoring method derived from the Avengers-Pro paper. Weave states that routing decisions per request are completed within 50ms, reducing costs by 40–70%.

Key Features and Design Characteristics

• Decision Unit (Action): The decision unit is individual API requests (Actions) sent by the agent to the top model, rather than sessions or turns. However, an option to pin the first routing result of a session for conversational consistency is enabled by default.
• Embedder Options: Users can choose between Jina AI's INT8 quantized model (jina-embeddings-v2-base-code) or a converted version of Qwen3-Embedding-0.6B. Both models are available for download from public repositories.
• Explicit Failure Handling: In cases of missing embedding models or timeouts, the system returns HTTP 503 instead of silently falling back to a default model, allowing immediate detection of issues (Failures are loud by design).
• Deployment and Usage: Built with Go 1.25 or higher, it can be operated via self-hosting or Weave's managed service. Installation scripts are provided for integration with major agent tools such as Claude Code and Codex CLI.