CrabTrap: An LLM-as-a-judge HTTP Proxy That Protects Agents in Production

Brex ran harnesses like OpenClaw in production, where agents needed to use real API keys, OAuth tokens, and service accounts, but faced the problem that hallucinations or prompt injection could cause them to send destructive requests. To solve this, they built CrabTrap, which intercepts every HTTP/HTTPS request an agent sends and uses LLM-as-a-judge to determine whether to allow it.

Existing approaches each had limitations.

• MCP gateways can only apply policy to MCP traffic.
• Provider-supplied guardrails are tied to specific models and hard to customize.
• Approaches like NVIDIA OpenShell are closer to sandbox egress control.

CrabTrap operates at the transport layer so it works independently of framework, language, and API. Setting HTTP_PROXY and HTTPS_PROXY in the agent's environment variables routes all outbound requests through CrabTrap, and for HTTPS it issues certificates with its own CA to perform TLS interception. If needed, the container's iptables can also be used to block direct connections outside the proxy.

Judgment happens in two stages.

1. Static rules first check exact, prefix, and glob matching on the URL along with HTTP method conditions. Deny rules always take priority, and this stage completes in microseconds using cached regexp.
2. If nothing matches, an LLM judge receives the full request context and the agent-specific natural-language policy, and returns ALLOW or DENY with a reason as JSON.

Policy authoring and validation are also automated.

• A policy builder analyzes past traffic, samples representative requests, and produces a draft policy that reflects actual behavior.
• An eval system replays past audit entries against the draft policy to show what would change.
• Results can be viewed broken down by method, URL, original verdict, and agreement status, and even thousands of entries can be replayed within minutes using parallel judge calls.
• All past requests are stored in PostgreSQL and can be queried via the admin API and web dashboard.

The judge's input is structured to prevent prompt injection. Requests are passed as JSON containing method, URL, headers, and body, with headers prioritizing security-relevant items and capped at 4KB total. Bodies are truncated at 16KB with a warning attached, and multipart content is converted into a summary of each part before being sent.

In operation, the policy builder matched human judgment at a high level using just a few days of real traffic, and many requests were absorbed by static rules — in one production case, LLM calls accounted for less than 3% of the total. The proxy was used not just as an enforcement tool but also as a discovery tool that reveals the noise and waste agents create.

Brex is open-sourcing CrabTrap, with plans to add SSO, fine-grained RBAC, a workflow for agents to request elevated permissions, and policy recommendations based on denial patterns.