The Invisible Work of Building Trustworthy AI Agents

The image of an AI agent booking travel on your behalf is compelling, but in reality, very small actions—scrolling, clicking, tapping, selecting dates behind pop-ups, recovering from broken forms—must be perfected first. Inside Amazon, this foundational skill set is called normcore agents, and the focus is on making agents extremely good at the most mundane yet most critical interactions needed to handle real-world software.

To this end, Amazon AGI Lab is building high-fidelity reinforcement learning (RL) 'gyms.' Each gym isolates a single skill for repeated training, varies conditions, induces failures, and instruments the results, designed to make agents resilient to the messiness of web interfaces and the unpredictability of legacy systems. These environments serve as the foundation for training not just travel booking, but the atomic actions that make up real work—entering addresses, selecting from dropdowns, multi-step workflows.

Reliability is central to this approach. Since an agent is not merely a model that produces outputs but an actor that changes the actual state of a system, it must be able to judge for itself whether the system responded correctly after an action. To enable this, the environments provide ground-truth signals such as DOM structure, UI timing, network behavior, and backend state changes, and each task is evaluated by a formal verifier that specifies success conditions, permitted changes, and changes that must never occur.

For example, a "send e-mail" task isn't judged simply by whether it looks like a button was clicked. It's only counted as a success if exactly one new e-mail record is created, with no other records modified or deleted. Repeating this process of providing verifiable reward signals thousands of times over is what turns a one-off success into a stable capability usable in actual operations.

Amazon is already training agents in gyms spanning dozens of domains and thousands of tasks. Examples include:

• calendar stability test: Selecting a date exactly once and correctly, and verifying the resulting backend state, even as UI elements shift due to zooming, re-rendering, or overlapping layers
• dropdown discipline drill: Learning that on-screen changes and actual system reflection can diverge, and trusting system state rather than the UI
• async endurance run: Maintaining consistency while enduring timing errors and intermittent failures in long asynchronous flows involving search, filtering, verification, and refreshing entangled together

The core message is clear: only when agents can reliably perform manipulations that seem trivial to humans can larger capabilities—like travel booking or business process automation—be built on top of them. Ultimately, trustworthy AI agents are built not first through 'intelligence,' but through training to get the small things right, all the way through.