FutureBench, a Benchmark for Predicting Future Events, Released

Existing AI benchmarks focus on past knowledge or already-solved problems, making them vulnerable to data contamination and leaderboard gaming.

FutureBench proposes a new benchmark that evaluates an AI agent's ability to predict future events such as geopolitical shifts, economic trends, and technology trends. Since the data for future predictions does not yet exist, data contamination is fundamentally impossible, and because outcomes become clear as time passes, objective verification is possible.

FutureBench uses the following approach:

• News-based question generation: Agents scrape and analyze major news sites to generate prediction questions about possible future scenarios.
• Use of prediction markets: It combines real prediction markets with news data to construct scenarios that require sophisticated reasoning rather than simple pattern matching.

This benchmark focuses on measuring whether agents can search for information and reason through complex causal relationships to generate real value.