Forge - A Tool That Boosts Agentic Task Success Rate of 8B Models from 53% to 99% with Guardrails

Forge is a reliability layer that stabilizes tool calls for self-hosted LLMs. It focuses on compensating for the frequent failures of small local models in multi-step agent workflows, raising execution success rates in agentic tasks.

The core features are as follows.

• Rescue parsing that recovers malformed tool calls
• Retry nudging that prompts retries on failure
• Required steps that enforce mandatory steps
• VRAM-aware token budget management
• Tiered context compression

In terms of performance, the top self-hosted configuration, Ministral-3 8B Instruct Q8 on llama-server, achieved 86.5% across 26 evaluation scenarios and 76% on the hardest tier. The post explains that Forge's guardrails and ablation study were compiled and published as Forge: A Reliability Layer for Self-Hosted LLM Tool-Calling, and states the license is MIT.

There are three ways to use it.

• WorkflowRunner: manages the entire agent loop, prompts, tool execution, context compression, and guardrails in an integrated way
• Guardrails middleware: inserted into an existing orchestration loop
• OpenAI-compatible proxy server: sits transparently between clients like opencode, Continue, aider and the local model server

The proxy automatically injects a synthetic respond tool into requests that have tools, so the model calls respond(message="...") instead of plain text, and strips it from the response so it appears to the client as normal text.

Supported backends are Ollama, llama-server (llama.cpp), Llamafile, and Anthropic. The post summarizes that llama-server offers the highest performance and control, Ollama offers easy setup, Llamafile offers single-binary execution, and Anthropic serves as a frontier baseline and hybrid workflows.

Installation is available via pip install forge-guardrails, and for the Anthropic client, additionally install pip install "forge-guardrails[anthropic]". Requirements are Python 3.12+ and a running LLM backend.

The evaluation harness measures multi-step tool-calling stability across model and backend combinations using 26 scenarios, divided into the OG-18 baseline tier and 8 advanced_reasoning tiers. Separately, it includes 865 deterministic unit tests that can run without an LLM backend, plus an evaluation harness targeting real backends.