PyMuPDF Outperforms Docling in Document Parsing Speed and Quality

E-Papyrus and Artifex, the developer of PyMuPDF, released results comparing the performance of the local document parsers PyMuPDF and Docling using ParseBench, LlamaIndex's public benchmark suite. To ensure benchmark fairness, this test used a third-party (competitor) benchmark rather than a self-evaluation by the developers, measuring a total of 2,078 documents on identical AWS instances.

Performance and Resource Comparison

Measurements in the same environment showed that PyMuPDF had an overall advantage over Docling.

• Overall Quality Score: PyMuPDF 53.6 vs Docling 50.72
• Table Extraction Score: PyMuPDF 72.0 vs Docling 69.0
• Visual Grounding Score: PyMuPDF 61.9 vs Docling 65.6
• Processing Speed: Based on median processing time per page, PyMuPDF was 0.47 seconds, while Docling was 11.0 seconds, making PyMuPDF significantly faster.
• Installation Size: PyMuPDF was 278MB, much lighter than Docling (1.6GB).

Implications and Limitations

PyMuPDF is advantageous for large-scale document parsing pipelines that require high processing speed and low memory usage in CPU-based environments. On the other hand, Docling recorded a higher score in visual grounding, but in this benchmark, PyMuPDF proved more efficient in terms of speed and lightweight nature. These results depend on specific versions (PyMuPDF 1.28.2, Docling 2.124.0) and ParseBench criteria, and additional verification based on data characteristics is required in actual work environments.