AI BriefingLlamaIndex releases ParseBench-2 with severity-weighted metrics and OCRia-1.5 models
·2026.11.12 09:00
Key point
The new benchmark uses 330+ error types to weight parsing errors by severity, correcting previous metrics that penalized formatting over factual errors.
Details
- LlamaIndex released ParseBench-2, a document parsing benchmark that introduces severity-weighted scoring to align deterministic metrics with human judgment of error impact.
- The dataset expanded from 2.1k to 4.3k cases, adding forms as a sixth evaluation dimension and checking every value across 1,025 charts (36,958 values total).
- LlamaParse Agentic Plus achieved the highest overall score of 89.8, outperforming the top direct vision model (Claude Opus 5.5 at 85.7) and the strongest competing document parser (Reducto at 83.9).
- Charts show the widest capability gap with scores ranging from 1.2 to 92.6, while visual grounding remains a challenge for frontier models with a median score of 64.5 compared to 83.8 for specialized parsers.
- OCRia-1.5 models power LlamaParse's Cost Effective, Agentic, and Agentic Plus modes, with Agentic Plus delivering 88.4 accuracy at $0.0281 per page, approximately 57% lower cost than the strongest direct vision model.
- The benchmark is open-source, with datasets, evaluation code, and the research paper available on Hugging Face and GitHub.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.