Offloading Bounded Decisions to Jev Cuts Agent LLM Calls by 29.6% on SOP-Bench

Offloading bounded decisions to a specialized model can significantly reduce the workload of a main LLM in agent harnesses. Testing Qwen 3.8 27B with hosted Jev on 360 SOP-Bench cases demonstrated a 29.6% reduction in main-LLM calls, dropping from 2,537 to 1,785.

Implementation Strategy

The experiment used the Proceda SOP-execution harness and tested two specific insertion points for the decision model:

• Step completion after tool calls: Jev verifies if necessary tool calls are made and evidence is sufficient. If the probability of completion reaches 0.8, the harness advances without a follow-up LLM turn.
• Bounded business decisions: For tasks like email intent, Jev receives the email, policy, and allowed outcomes. A valid choice with probability ≥ 0.8 is accepted; otherwise, the decision falls back to Qwen.

Performance Metrics

With both changes enabled, the system achieved the following results compared to the Qwen-only baseline:

• Main-LLM calls: Fell from 2,537 to 1,785 (29.6% fewer).
• Execution API cost: Dropped from $6.41 to $4.70 (26.6% lower), including Jev costs.
• Mean case latency: Reduced from 8.64s to 6.94s (19.7% lower).
• Task accuracy: Achieved 360/360 correct tasks, compared to 357/360 for Qwen alone.

Tradeoffs and Limitations

Completion checking alone saved 23.8% of LLM calls but resulted in 356/360 correct tasks, slightly below the baseline of 357/360. The combined result showing perfect accuracy is specific to this 360-case subset and does not establish a general accuracy improvement. Additionally, including the final structured-plan preparation cost reduces the estimated cost saving to 24.6%.