AI Reasoning: Planning vs Prediction

Current LRMs reinforce internal thinking tokens and Chain of Thought (CoT) to find a path to the correct answer, but the authors view this as merely an extension of next-token prediction. Learning only the end-point reward via RL and exploring long chains at inference time leads to inefficient per-input resource allocation, and as chains get longer, the search space and cost grow rapidly.

The key limitations the authors point out are as follows.

• Since reward is only given at the end of the chain, learning intermediate steps is difficult.
• Transparency, predictability, and customizability are lacking.
• As input size and difficulty increase, generalization can break down quickly.
• Fine-grained control over parallelization and resource adaptation is difficult.

The alternative is planning, which searches in the space of action sequences rather than tokens. An external controller orchestrates the LLM along with search, DB calls, and tool use, and plans should model both cost and probability of success together. During the training phase, planning methods should be learned, and during the test phase, prior plans should be concretized according to the situation, alternating between execution and planning. Monte Carlo Tree Search (MCTS) is mentioned as a representative technique for stochastic environments, and each step should be validated with a verifier.

AI21 presents as core principles: a restricted Python subset language for expressing plans, clear goal definition, decision-theoretic planning, anytime execution, parallelism, human in the loop, and rejection when necessary. Finally, through the AI21 Maestro case study, it introduces improvements on IFEval (with LLM-as-a-Judge and code-based verifiers attached): GPT-4o improved from about 85% to 91.9%, Claude Sonnet 3.5 from about 88% to 95.2%, and o3-mini from about 92% to 95.7%.