Agent Substrate: Kubernetes-based Execution Environment for Isolated AI Agents

Overview and Architecture

Agent Substrate is an early-stage, Apache-2.0 licensed project designed to run AI agents in isolated Kubernetes environments. It aims to reduce idle resource consumption by managing agent lifecycles through a control plane, node supervisors, and a networking stack. The system supports both gVisor (ateworker-gvisor) and micro-VM (ateworker-microvm) based sandboxing, allowing for secure execution of arbitrary code while preserving state across suspend/resume cycles.

Key Features and Demos

The project demonstrates several core capabilities through specific demos:

• State Preservation: A counter demo shows state persistence during suspension and on-demand resumption via the Substrate router.
• Secure Sandboxing: An Alpine Linux-based environment allows arbitrary shell execution with filesystem state preserved between sessions.
• Resource Multiplexing: A demo multiplexes multiple Claude Code agents onto a limited worker pool, effectively oversubscribing physical hardware.
• Request Parking: The router holds inbound requests when workers are unavailable, preventing 503 errors in oversubscribed pools.
• Autoscaling: Worker pools scale based on assigned worker counts using Prometheus-adapter and HPA.

Deployment and Requirements

Local development requires Go, kubectl, and docker, with kind managed automatically. Running over 1000 sandboxes on a Linux host requires adjusting kernel neighbor table settings (net.ipv4.neigh.default.gc_thresh1/2/3 to 4096/16384/32768). GKE deployment involves bootstrapping infrastructure (GKE cluster, GCS, IAM) and using specific scripts for system and demo installation. The default credential provider uses Kubernetes service accounts with default-deny namespace policies, requiring explicit permissions for actors to inject credentials.