Orchestrating Modular AI Agents

Complex agent systems don't reveal their failure points from the final answer alone. Whether the problem occurred during retrieval, routing, or synthesis is obscured if you only look at the output.

Levelt(1989)'s model of human language production offers a useful analogy for this problem. Humans decide what to say during conceptualization, convert it into linguistic structure during formulation, and turn it into actual output during articulation. Each stage is a module with separated input and output, allowing independent evaluation.

AI agents are evolving in the same direction. While loops like ReAct repeat reasoning-action-observation, AI21 Maestro separates reasoning, planning, and execution to provide higher modularity. This structure reduces context drift, preserves intent through to the execution stage, and allows each action's result to be externally confirmed by a verifier.

Evaluation is also divided by stage. Even within the same system, a conversational request like "I'm so exhausted today" and an investigative request like "Has the office pet policy changed?" need to take different paths. Routing can be verified by classification accuracy, retrieval by grounding—such as whether the actual current and past HR policy documents were correctly retrieved—and synthesis by faithfulness and coverage.

When responsibilities are separated this way, incorrect routing and ungrounded synthesis no longer have to be lumped together as the same failure. Problems can be localized to a specific component and fixed without having to rework the entire system. The article concludes that what's ultimately needed is not a better global score but a clearer structure. Agents with separated reasoning, planning, execution, and verification are easier to audit and diagnose, making them systems that are easier to systematically improve.