Ollama Multi-Machine Routing Tool Stoke Released

When running Ollama across multiple machines, Stoke has been released to automatically distribute and manage requests. This tool is a 5.5 MB Rust binary that works without any separate runtime dependencies.

The main features are as follows:

• Node-aware routing: It polls each node's /api/tags and /api/ps to preferentially select Warm Nodes where the model is already loaded, and automatically performs failover on failure.
• Federation: By running Stoke on each machine, multiple Ollama instances can be managed together through a single network endpoint (including Auth).
• Budget Caps: In case cloud providers are added, USD spending limits can be set per API key.
• Loop Detection: Prevents agents from falling into retry loops where they repeat identical or similar requests.
• Auto-routing and Hedged Dispatch: Selects the optimal model and node based on cost, latency, and preference, and supports a feature that sends small prompts simultaneously to idle machines and uses the result from whichever machine responds first.