Scaling Managed Agents: Separating the Brain from the Hands

Managed Agents separates an agent's core components into session, harness, and sandbox, designed so that the interface stays stable for a long time even as implementations change. By breaking the harness out of its structure bound inside a container, issues with infrastructure failures, session recovery, VPC integration, and security boundaries were all resolved together.

Initially, all components were put into a single container, but as a result, the server became like a pet. When the container died, the session died with it, and with only the WebSocket event stream, it was hard to tell whether the problem was the harness, the network, or the container going down.

After restructuring, the harness no longer treats the container directly as an execution environment, but only accesses it through a tool call like execute(name, input) → string. If the container fails, the harness receives this as a tool-call error and returns it to Claude, and if needed, can spin up a new container again with provision({resources}).

The harness itself was also made into cattle. Since session logs live outside the harness, even if the harness dies, it can be restarted with wake(sessionId), read the event log from getSession(id), and continue on with emitEvent(id, event).

Separation was also important from a security standpoint. Previously, when untrusted code ran in the same container as Claude, prompt injection could reach environment variables and tokens, but now tokens are kept outside the sandbox. Git performs only the initial clone using a repo access token and connects only to the local remote, and for custom tools, a proxy handles OAuth tokens on their behalf through MCP and a vault.

The session is also separated from Claude's context window. Instead of irreversible summarization like compaction or trimming, events are permanently stored in the session log, and positional slices are retrieved using getEvents(). The harness can reprocess these events as needed to perform prompt cache efficiency or context engineering.

This separation also enabled scaling to many brains and many hands. Since there was no longer a need to spin up containers in advance, inference could start as soon as a pending event was read, and as a result, p50 TTFT dropped by about 60%, and p95 by over 90%. Meanwhile, each hand is treated as a simple tool, allowing Claude to handle different execution environments — such as containers, phones, and a Pokémon emulator — in the same way.