Technical Overview of Maestro

Maestro is a dynamic planning system introduced by AI21 that takes instructions and requirements as separate inputs and verifies the output throughout the generation process. Requirements cover content, style, format, genre, perspective, and guardrails, and the system returns a score and reasons for non-compliance for each item. It also connects to built-in file search (a RAG system) and web search, so it can be attached to complex generation and retrieval pipelines.

Execution doesn't end with a single generation. Using inference-time compute, it finds a path that satisfies both quality criteria and budget while continuously refining the current candidate. Each call creates a tree made up of LLM calls and tool calls, and the plan includes Best-of-N and generate and fix. It also understands code dependencies to coordinate parallel and sequential tasks.

• Faster work: you can get quality guarantees within budget without having to design complex production workflows yourself.
• Smarter fixes: when a failure pattern appears in the output, you can add a new requirement to correct it immediately. For example, an escaping issue with strings containing apostrophes was blocked by adding it as a requirement.

Evaluation was conducted along two axes. On complex multi-hop RAG problems from the FRAMES benchmark, Maestro outperformed existing agent systems where an LLM makes decisions at runtime. On IFEval, verification was done in two ways.

• LLM-as-a-Judge: converting verification functions into semantic requirements
• Code validation: implementing verification logic as a program and using it directly for iterative refinement

The proprietary dataset Requirements Following includes grounding tasks such as question answering and summarization, along with strong constraints such as a 50-word limit, covering only listening, italicizing leader(s)/manager(s), and prohibiting mentions of specific people.

On this dataset, Maestro improved both requirement satisfaction rate and accuracy, showing gains even on models with high baseline scores like o3. In blind evaluations by human experts, it also had a higher rate of 'very satisfied' and a lower rate of 'unsatisfied' compared to standalone gpt-4o calls. In conclusion, Maestro is a system that explicitly defines requirements and verifies and corrects them during execution, raising the quality of complex LLM tasks within budget.