MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios

• URL: http://arxiv.org/abs/2603.28130v1
• Date: Mon, 30 Mar 2026 07:47:46 GMT
• Title: MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios
• Abstract summary: We introduce Multilingual Document Parsing Benchmark, the first benchmark for multilingual digital and photographed document parsing.<n> MDPBench comprises 3,400 document images spanning 17 languages, diverse scripts, and varied photographic conditions.
• Score: 72.8160644291677
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Multilingual Document Parsing Benchmark, the first benchmark for multilingual digital and photographed document parsing. Document parsing has made remarkable strides, yet almost exclusively on clean, digital, well-formatted pages in a handful of dominant languages. No systematic benchmark exists to evaluate how models perform on digital and photographed documents across diverse scripts and low-resource languages. MDPBench comprises 3,400 document images spanning 17 languages, diverse scripts, and varied photographic conditions, with high-quality annotations produced through a rigorous pipeline of expert model labeling, manual correction, and human verification. To ensure fair comparison and prevent data leakage, we maintain separate public and private evaluation splits. Our comprehensive evaluation of both open-source and closed-source models uncovers a striking finding: while closed-source models (notably Gemini3-Pro) prove relatively robust, open-source alternatives suffer dramatic performance collapse, particularly on non-Latin scripts and real-world photographed documents, with an average drop of 17.8% on photographed documents and 14.0% on non-Latin scripts. These results reveal significant performance imbalances across languages and conditions, and point to concrete directions for building more inclusive, deployment-ready parsing systems. Source available at https://github.com/Yuliang-Liu/MultimodalOCR.

Related papers

• MonkeyOCRv2: A Visual-Text Foundation Model for Document AI [71.06391669976786]
  MonkeyOCRv2 is a visual-text pretrained model for document AI.<n> MonkeyDoc v2 is the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages.
  arXiv  Detail & Related papers  (2026-07-13T13:43:39Z)
• MORE: A Multilingual Document Parsing Benchmark and Evaluation [13.789730225958877]
  MORE is a large-scale benchmark designed for multilingual document parsing evaluation.<n>It covers 149 languages, making it the most linguistically diverse benchmark to date.<n>All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline.
  arXiv  Detail & Related papers  (2026-07-03T04:59:12Z)
• MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing [74.84107522458798]
  MPDocBench-Parse is a benchmark for multi-page document parsing in real-world applications.<n>It contains 433 manually annotated documents with 3,246 pages, covering 15 document types in English and Chinese, with diverse layout styles.
  arXiv  Detail & Related papers  (2026-05-21T07:36:41Z)
• Dolphin-v2: Universal Document Parsing via Scalable Anchor Prompting [46.102790941920865]
  We present Dolphin-v2, a two-stage document image parsing model that substantially improves upon the original Dolphin.<n>In the first stage, Dolphin-v2 jointly performs document type classification (digital-born versus photographed) alongside layout analysis.<n>In the second stage, we employ a hybrid parsing strategy: photographed documents are parsed holistically as complete pages to handle geometric distortions, while digital-born documents undergo element-wise parallel parsing guided by the detected layout anchors.
  arXiv  Detail & Related papers  (2026-02-05T07:09:57Z)
• VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents [45.37806172594631]
  multimodal image editing models have achieved substantial progress, enabling users to manipulate visual content in a flexible and interactive manner.<n> visual document image editing involves modifying textual content within images while preserving the original text style and background context.<n>Existing approaches, including AnyText, GlyphControl, and TextCtrl, predominantly focus on English-language scenarios and documents with relatively sparse textual layouts.
  arXiv  Detail & Related papers  (2026-01-27T16:51:05Z)
• DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation [18.531174868051558]
  We introduce DocPTBench, a benchmark specifically designed for Photographed Document Parsing and Translation.<n>Our experiments demonstrate that transitioning from digital-born to photographed documents results in a substantial performance decline.<n>This substantial performance gap underscores the unique challenges posed by documents captured in real-world conditions.
  arXiv  Detail & Related papers  (2025-11-23T13:02:11Z)
• WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild? [64.62909376834601]
  This paper introduces WildDoc, the inaugural benchmark designed specifically for assessing document understanding in natural environments.<n> evaluation of state-of-the-art MLLMs on WildDoc expose substantial performance declines and underscore the models' inadequate robustness compared to traditional benchmarks.
  arXiv  Detail & Related papers  (2025-05-16T09:09:46Z)
• MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding [66.23502779435053]
  Large Vision-Language Models (LVLMs) have achieved remarkable performance in many vision-language tasks. Existing benchmarks either contain limited fine-grained evaluation samples mixed with other data, or are confined to object-level assessments in natural images. We propose using document images with multi-granularity and multi-modal information to supplement natural images.
  arXiv  Detail & Related papers  (2024-10-25T16:00:55Z)
• OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text [112.60163342249682]
  We introduce OmniCorpus, a 10 billion-scale image-text interleaved dataset. Our dataset has 15 times larger scales while maintaining good data quality. We hope this could provide a solid data foundation for future multimodal model research.
  arXiv  Detail & Related papers  (2024-06-12T17:01:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.