Performance Improvement Strategy Through Multiple Agent Runs

In complex agentic tasks, instead of improving the model itself or scaling up resources, re-running the same task multiple times can be a more effective way to improve performance.

This is because it offsets the random errors that occur during each run, while preserving the accurate information that is commonly derived from multiple runs' results.

Key Data and Cases:

• Forecasting Benchmark: In a test of 1,367 questions by FutureSearch.ai, combining the run results of Claude, Gemini, and GPT improved the score to 0.125, compared to a single Claude Opus run (Brier score 0.130).
• Context Supplementation: In a case forecasting Brazil's climate legislation, two runs that took different search paths showed the effect of supplementing each other's missing context (e.g., information about the COP30 hosting).

Implementation Strategy:

• For one-off tasks, directly compare the two results and select the best answer.
• For recurring tasks, it is efficient to deploy a dedicated second agent that reviews and reconciles the two outputs.
• If the cost is affordable, it is recommended to apply this method first before investing in fundamentally improving the agent's quality.