NVIDIA Maestro

Enterprise AI shows promise at the PoC stage, but often fails to move into production because it cannot reliably handle reasoning, planning, and execution. AI21 has integrated Maestro with NVIDIA NIM, presenting a self-hosted AI setup that enterprises can operate directly.

Simply throwing prompts at a model lacks consistency and verification, while hardcoded chains lack flexibility. Maestro is an AI Planning & Orchestration System that dynamically orchestrates multi-step workflows, selecting the right model for the task, verifying decisions, and then connecting through to execution.

The reasons enterprises seek self-hosted AI are clear.

• Data sovereignty: Sensitive data, as in finance, healthcare, and government, must stay within internal environments.
• Performance and cost: On-premises operation reduces unpredictable cloud costs and optimizes GPU utilization.
• Control and customization: AI can be tailored to each organization's workflows.
• Regulatory compliance: Deployment via private cloud, on-prem, or VPC can meet security and compliance requirements.

NVIDIA NIM microservices provide low-latency, high-efficiency inference and pre-optimized containers, while Maestro adds observability and control. This lets enterprises split work by task—using an LLM for text generation, an embedding model for search, and a reasoning model for structured judgment—while pursuing automated financial risk analysis, improved customer response, and the building of AI agents that carry out complex tasks.

NVIDIA's Amanda Saunders said that high-performance inference in production environments underpins AI agents and reasoning, while AI21's Ori Goshen emphasized that what must be trusted is not response generation but the execution of complex tasks.