Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training

• URL: http://arxiv.org/abs/2603.23885v2
• Date: Fri, 27 Mar 2026 06:38:20 GMT
• Title: Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training
• Abstract summary: We propose a data-training co-design framework for robust end-to-end document parsing.<n>Our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios.
• Score: 23.87978106727431
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Traditional cascaded pipelines depend on precise layout analysis and often fail under casually captured or non-standard conditions. Although end-to-end approaches mitigate this dependency, they still exhibit repetitive, hallucinated, and structurally inconsistent predictions - primarily due to the scarcity of large-scale, high-quality full-page (document-level) end-to-end parsing data and the lack of structure-aware training strategies. To address these challenges, we propose a data-training co-design framework for robust end-to-end document parsing. A Realistic Scene Synthesis strategy constructs large-scale, structurally diverse full-page end-to-end supervision by composing layout templates with rich document elements, while a Document-Aware Training Recipe introduces progressive learning and structure-token optimization to enhance structural fidelity and decoding stability. We further build Wild-OmniDocBench, a benchmark derived from real-world captured documents for robustness evaluation. Integrated into a 1B-parameter MLLM, our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios. All models, data synthesis pipelines, and benchmarks will be publicly released to advance future research in document understanding.

Related papers

• NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Documents [18.60763757268658]
  We propose NaviDC-OCR, a unified framework for document parsing.<n> NaviDC-OCR introduces deformation-aware learning to incorporate geometric perception into VLMs.<n>It achieves state-of-the-art performance across diverse document parsing benchmarks.
  arXiv  Detail & Related papers  (2026-08-13T07:34:21Z)
• DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding [63.257540233507626]
  We propose a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow.<n>We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems.
  arXiv  Detail & Related papers  (2026-04-14T14:39:26Z)
• FireRed-OCR Technical Report [30.019999826760003]
  We introduce FireRed-OCR, a framework to transform general-purpose VLMs into pixel-precise structural document parsing experts.<n>To address the scarcity of high-quality structured data, we construct a Geometry + Semantics'' Data Factory.<n>We propose a Three-Stage Progressive Training strategy that guides the model from pixel-level perception to logical structure generation.
  arXiv  Detail & Related papers  (2026-03-02T13:19:23Z)
• Training-Free Acceleration for Document Parsing Vision-Language Model with Hierarchical Speculative Decoding [102.88996030431662]
  We propose a training-free and highly efficient acceleration method for document parsing tasks.<n>Inspired by speculative decoding, we employ a lightweight document parsing pipeline as a draft model to predict batches of future tokens.<n>We demonstrate the effectiveness of our approach on the general-purpose OmniDocBench.
  arXiv  Detail & Related papers  (2026-02-13T14:22:10Z)
• MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patterns [80.05126590825121]
  MonkeyOCR v1.5 is a unified vision-language framework that enhances both layout understanding and content recognition.<n>To address complex table structures, we propose a visual consistency-based reinforcement learning scheme.<n>Two specialized modules, Image-Decoupled Table Parsing and Type-Guided Table Merging, are introduced to enable reliable parsing of tables.
  arXiv  Detail & Related papers  (2025-11-13T15:12:17Z)
• Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing [46.14775667559124]
  Document parsing from scanned images remains a significant challenge due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables.<n>Existing supervised fine-tuning methods often struggle to generalize across diverse document types, leading to poor performance, particularly on out-of-distribution data.<n>We introduce LayoutRL, a reinforcement learning framework that optimize layout understanding through composite rewards integrating normalized edit distance count accuracy, and reading order preservation.<n>We show that Infinity-Bench consistently achieves state-of-the-art performance across a broad range of document types, languages, and structural complexities.
  arXiv  Detail & Related papers  (2025-10-17T06:26:59Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing [46.14775667559124]
  layoutRL is an end-to-end reinforcement learning framework that trains models to be explicitly layout-aware.<n>We will publicly release our code and dataset to accelerate progress in robust document understanding.
  arXiv  Detail & Related papers  (2025-06-01T15:19:52Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.