Karpathy's Leash

Karpathy warned against the hype around fully autonomous AI agents, saying that AI should be controlled, kept on a leash. The key isn't whether AI acts, but whether it acts consistently within a trustworthy range.

Large LLMs can seem superhuman one moment and then make mistakes no human would make the next. Especially in enterprise workflows involving multiple tools, databases, and decisions, small errors compound exponentially at every step.

This is why the prompt-and-pray approach of throwing a complex request in one shot has clear limits. Instead, a neuro-symbolic hybrid structure is needed. The language model handles step-by-step reasoning, while symbolic logic controls the overall execution, ensuring structure and verification.

• Break down tasks into structured execution plans.
• Evaluate multiple approaches in parallel at each step.
• Verify requirements individually with the language model and deterministic code, and auto-correct when necessary.
• Reflect compute budget and latency constraints together.
• Leave an auditable execution record alongside the final result.

This approach is especially useful for tasks like financial due diligence, which require analyzing hundreds of documents and cross-verifying facts. AI21 implemented this in AI21 Maestro, and emphasizes that the conclusion lies not in increasing autonomy, but in securing controllable reliability.