How to Customize Agent Harnesses Using Middleware

Agent Harness is the core element that connects an LLM to external environments such as data, memory, and tools, turning it into an executable system. The basic structure takes the simple form of an LLM looping through tool calls.

With the existing approach, it was easy to modify prompts or the tools themselves, but changing the agent's Core Loop was tricky. For example, it was difficult to force a specific step to run before model execution, or to insert logic that validates a tool's output.

Agent Middleware solves this problem by providing various Hooks that let you run custom logic at each stage of the loop.

• before_agent: Runs once on invocation (memory loading, input validation, etc.)
• before_model: Runs before the model call (history summarization, PII detection, etc.)
• wrap_model_call: Wraps the entire model call step (caching, retries, dynamic model requests, etc.)
• wrap_tool_call: Wraps tool execution (context injection, result interception, etc.)
• after_model: Runs after the model response but before tool execution (well suited for implementing Human-in-the-loop)
• after_agent: Runs once on completion (saving results, sending notifications, etc.)

Middleware can be used for a variety of use cases, including business logic and compliance (PII masking), dynamic agent control (runtime tool selection), and context management (summarization when hitting token limits).