Show HN: Forge – Guardrails Boost Agentic Task Performance of 8B Models from 53% to 99%

Forge is a Python library that improves tool-calling reliability for self-hosted LLMs. Through guardrails (rescue parsing, retry nudges, step enforcement) and context management (VRAM-based budgeting, hierarchical compression), it brings 8B local models up to top-tier performance in multi-step agentic workflows.

Key performance: The Ministral-3 8B Instruct Q8 + llama-server combination achieved 86.5% across a 26-scenario evaluation, and 76% on the hardest-tier tasks.

Three ways to use it

• WorkflowRunner: Manages the full lifecycle from tool definitions to the agent loop
• Guardrail middleware: Drop the reliability stack into an existing orchestration loop
• Proxy server: An OpenAI-compatible proxy that transparently applies guardrails to any client (python -m forge.proxy)

Supports Ollama, llama-server (llama.cpp), Llamafile, and Anthropic backends. The proxy automatically injects a respond tool to help small models correctly choose between text and tool calls.

Validation: Includes 865 unit tests and a 26-scenario evaluation harness. Distributed on PyPI, with an accompanying paper published in an ACM journal.