Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing

• URL: http://arxiv.org/abs/2506.03197v3
• Date: Tue, 21 Oct 2025 02:15:23 GMT
• Title: Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
• Authors: Baode Wang, Biao Wu, Weizhen Li, Meng Fang, Zuming Huang, Jun Huang, Haozhe Wang, Yanjie Liang, Ling Chen, Wei Chu, Yuan Qi,
• Abstract summary: layoutRL is an end-to-end reinforcement learning framework that trains models to be explicitly layout-aware.<n>We will publicly release our code and dataset to accelerate progress in robust document understanding.
• Score: 46.14775667559124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated parsing of scanned documents into richly structured, machine-readable formats remains a critical bottleneck in Document AI, as traditional multi-stage pipelines suffer from error propagation and limited adaptability to diverse layouts. We introduce layoutRL, an end-to-end reinforcement learning framework that trains models to be explicitly layout-aware by optimizing a composite reward of normalized edit distance, paragraph count accuracy, and reading order preservation. Leveraging our newly released dataset, Infinity-Doc-55K, which combines 55K high-fidelity synthetic scanned document parsing data with expert-filtered real-world documents, we instantiate layoutRL in a vision-language-model-based parser called Infinity-Parser. Evaluated on English and Chinese benchmarks for OCR, table and formula extraction, and reading order detection, Infinity-Parser achieves new state-of-the-art performance in both accuracy and structural fidelity, outpacing specialist pipelines and general-purpose vision-language models. We will publicly release our code and dataset to accelerate progress in robust document understanding.

Related papers

• Training-Free Acceleration for Document Parsing Vision-Language Model with Hierarchical Speculative Decoding [102.88996030431662]
  We propose a training-free and highly efficient acceleration method for document parsing tasks.<n>Inspired by speculative decoding, we employ a lightweight document parsing pipeline as a draft model to predict batches of future tokens.<n>We demonstrate the effectiveness of our approach on the general-purpose OmniDocBench.
  arXiv  Detail & Related papers  (2026-02-13T14:22:10Z)
• Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decoding [35.429403152845836]
  Youtu-Parsing is an efficient and versatile document parsing model designed for high-performance content extraction.<n>The model exhibits strong robustness when handling rare characters, multilingual text, and handwritten content.<n>Youtu-Parsing achieves state-of-the-art (SOTA) performance on the OmniDocBench and olmOCR-bench benchmarks.
  arXiv  Detail & Related papers  (2026-01-28T09:37:13Z)
• MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patterns [80.05126590825121]
  MonkeyOCR v1.5 is a unified vision-language framework that enhances both layout understanding and content recognition.<n>To address complex table structures, we propose a visual consistency-based reinforcement learning scheme.<n>Two specialized modules, Image-Decoupled Table Parsing and Type-Guided Table Merging, are introduced to enable reliable parsing of tables.
  arXiv  Detail & Related papers  (2025-11-13T15:12:17Z)
• Automated Invoice Data Extraction: Using LLM and OCR [0.0]
  This work introduces a holistic Artificial Intelligence (AI) platform combining OCR, deep learning, Large Language Models (LLMs) and graph analytics to achieve unprecedented extraction quality and consistency.
  arXiv  Detail & Related papers  (2025-11-01T19:05:09Z)
• Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing [46.14775667559124]
  Document parsing from scanned images remains a significant challenge due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables.<n>Existing supervised fine-tuning methods often struggle to generalize across diverse document types, leading to poor performance, particularly on out-of-distribution data.<n>We introduce LayoutRL, a reinforcement learning framework that optimize layout understanding through composite rewards integrating normalized edit distance count accuracy, and reading order preservation.<n>We show that Infinity-Bench consistently achieves state-of-the-art performance across a broad range of document types, languages, and structural complexities.
  arXiv  Detail & Related papers  (2025-10-17T06:26:59Z)
• Logics-Parsing Technical Report [8.982345117231661]
  We propose Logics-Parsing: an end-to-end LVLM-based model augmented with reinforcement learning.<n>Our model incorporates meticulously designed reward mechanisms to optimize complex layout analysis and reading order inference.<n>We introduce LogicsParsingBench, a curated set of 1,078 page-level PDF images spanning nine major categories and over twenty sub-categories.
  arXiv  Detail & Related papers  (2025-09-24T04:54:37Z)
• VDInstruct: Zero-Shot Key Information Extraction via Content-Aware Vision Tokenization [7.769156392417315]
  Key Information Extraction underpins the understanding of visual documents.<n>Existing multimodal large language models (MLLMs) often perform poorly on dense documents.<n>We introduce VDInstruct, an MLLM that separates spatial region detection from semantic feature extraction.
  arXiv  Detail & Related papers  (2025-07-13T08:15:11Z)
• SCAN: Semantic Document Layout Analysis for Textual and Visual Retrieval-Augmented Generation [5.458935851230595]
  We present SCAN, a novel approach enhancing both textual and visual Retrieval-Augmented Generation (RAG) systems.<n>SCAN uses a coarse-grained semantic approach that divides documents into coherent regions covering continuous components.<n>Our experimental results across English and Japanese datasets demonstrate that applying SCAN improves end-to-end textual RAG performance by up to 9.0% and visual RAG performance by up to 6.4%.
  arXiv  Detail & Related papers  (2025-05-20T14:03:24Z)
• VISTA-OCR: Towards generative and interactive end to end OCR models [3.7548609506798494]
  VISTA-OCR is a lightweight architecture that unifies text detection and recognition within a single generative model.<n>Built on an encoder-decoder architecture, VISTA-OCR is progressively trained, starting with the visual feature extraction phase.<n>To enhance the model's capabilities, we built a new dataset composed of real-world examples enriched with bounding box annotations and synthetic samples.
  arXiv  Detail & Related papers  (2025-04-04T17:39:53Z)
• OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models [58.45517851437422]
  Visually-situated text parsing (VsTP) has recently seen notable advancements, driven by the growing demand for automated document understanding.<n>Existing solutions often rely on task-specific architectures and objectives for individual tasks.<n>In this paper, we introduce Omni V2, a universal model that unifies VsTP typical tasks, including text spotting, key information extraction, table recognition, and layout analysis.
  arXiv  Detail & Related papers  (2025-02-22T09:32:01Z)
• HAND: Hierarchical Attention Network for Multi-Scale Handwritten Document Recognition and Layout Analysis [21.25786478579275]
  Handwritten document recognition is one of the most challenging tasks in computer vision.<n>Traditionally, this problem has been approached as two separate tasks, handwritten text recognition and layout analysis.<n>This paper introduces HAND, a novel end-to-end and segmentation-free architecture for simultaneous text recognition and layout analysis tasks.
  arXiv  Detail & Related papers  (2024-12-25T20:36:29Z)
• Detecting Document-level Paraphrased Machine Generated Content: Mimicking Human Writing Style and Involving Discourse Features [57.34477506004105]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>We introduce novel methodologies and datasets to overcome these challenges.<n>We propose MhBART, an encoder-decoder model designed to emulate human writing style.<n>We also propose DTransformer, a model that integrates discourse analysis through PDTB preprocessing to encode structural features.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• CREPE: Coordinate-Aware End-to-End Document Parser [13.530212337717515]
  We formulate an OCR-free sequence generation model for visual document understanding (VDU) Our model not only parses text from document images but also extracts the spatial coordinates of the text based on the multi-head architecture. Named as Coordinate-aware End-to-end Document. CREPE, our method uniquely integrates these capabilities by introducing a special token for OCR text.
  arXiv  Detail & Related papers  (2024-05-01T00:30:13Z)
• Visually Guided Generative Text-Layout Pre-training for Document Intelligence [51.09853181377696]
  We propose visually guided generative text-pre-training, named ViTLP. Given a document image, the model optimize hierarchical language and layout modeling objectives to generate the interleaved text and layout sequence. ViTLP can function as a native OCR model to localize and recognize texts of document images.
  arXiv  Detail & Related papers  (2024-03-25T08:00:43Z)
• mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding [55.4806974284156]
  Document understanding refers to automatically extract, analyze and comprehend information from digital documents, such as a web page. Existing Multi-model Large Language Models (MLLMs) have demonstrated promising zero-shot capabilities in shallow OCR-free text recognition.
  arXiv  Detail & Related papers  (2023-07-04T11:28:07Z)
• End-to-End Information Extraction by Character-Level Embedding and Multi-Stage Attentional U-Net [0.9137554315375922]
  We propose a novel deep learning architecture for end-to-end information extraction on the 2D character-grid embedding of the document. We show that our model outperforms the baseline U-Net architecture by a large margin while using 40% fewer parameters.
  arXiv  Detail & Related papers  (2021-06-02T05:42:51Z)
• Rethinking Text Line Recognition Models [57.47147190119394]
  We consider two decoder families (Connectionist Temporal Classification and Transformer) and three encoder modules (Bidirectional LSTMs, Self-Attention, and GRCLs) We compare their accuracy and performance on widely used public datasets of scene and handwritten text. Unlike the more common Transformer-based models, this architecture can handle inputs of arbitrary length.
  arXiv  Detail & Related papers  (2021-04-15T21:43:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.