AI Can't Read Investor Decks

The latest AI models show high performance on benchmarks, but exhibit severe performance degradation when handling real-world financial documents that include charts and graphs.

GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6 were tested on 25 tasks using practical financial documents such as earnings reports, investor decks, and regulatory fee schedules. The experiment was conducted in two formats: providing information as text only, and providing it as images (multimodal).

As a result, the models showed high accuracy of 72-80% on text-based data, but accuracy dropped sharply to 56-64% on image-based data. A consistent performance decline of about 16-20 percentage points in accuracy when images were provided was observed across models.

The major failure types are as follows:

• Misreading numerical values in dense visual documents
• Applying incorrect financial calculations even when input values are accurate

This shows that visual information extraction (Visual Extraction) remains a key bottleneck even for the latest frontier models.