GutenOCR: A Grounded Vision-Language Front-End for Documents

• URL: http://arxiv.org/abs/2601.14490v2
• Date: Thu, 22 Jan 2026 18:58:24 GMT
• Title: GutenOCR: A Grounded Vision-Language Front-End for Documents
• Abstract summary: We introduce a family of grounded OCR front-ends obtained by fine-tuning Qwen2.5-VL-3B and Qwen2.5-VL-7B.<n>The resulting single-checkpoint vision-language models expose reading, detection, and grounding through a unified, prompt-based interface.<n>We show that GutenOCR-7B more than doubles the composite grounded OCR score of its Qwen2.5-VL-7B backbone on 10.5K held-out business and scientific pages.
• Score: 0.42776193697639947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: GutenOCR is a family of grounded OCR front-ends obtained by fine-tuning Qwen2.5-VL-3B and Qwen2.5-VL-7B. The resulting single-checkpoint vision-language models expose reading, detection, and grounding through a unified, prompt-based interface. Trained on business documents, scientific articles, and synthetic grounding data, the models support full-page and localized reading with line- and paragraph-level bounding boxes and conditional ``where is x?'' queries. We introduce a grounded OCR evaluation protocol and show that GutenOCR-7B more than doubles the composite grounded OCR score of its Qwen2.5-VL-7B backbone on 10.5K held-out business and scientific pages (0.40 to 0.82). On Fox and OmniDocBench v1.5, our approach substantially improves region- and line-level OCR as well as text-detection recall, but reveals trade-offs in page-level linearization, color-guided OCR, and formula-heavy layouts.

Related papers

• How Far Can Synthetic Data Take Thai OCR? [3.8641218447429657]
  We build a Thai OCR model adapted without OCR labels from real Thai document pages.<n>Synthetic data provide exact labels at scale, but "realism" conflates source domain, page context, typography, spatial structure, and glyph variation.<n>We evaluate each variant under page- and crop-level training on printed and handwritten Thai documents.
  arXiv  Detail & Related papers  (2026-09-03T09:46:19Z)
• TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents [50.64307290680222]
  TongGuOCR is a layout-aware and token-augmented multimodal large language model (MLLM) for OCR of Chinese historical documents.<n>TongGuOCR achieves 93.76 AR and reduces NED from 10.43 to 6.15 and RO-ED from 7.53 to 3.49 relative to the best competing score for each metric.
  arXiv  Detail & Related papers  (2026-08-08T04:50:00Z)
• Cross-Temporal Sinhala OCR: Page-Level Adaptation and Diachronic Analysis [0.0]
  There are no publicly available real-world datasets for page-level Sinhala OCR.<n>We introduce sinhala-ocr-lk-acts-1010, an annotated dataset of 1,010 page-level images and their transcriptions.<n>Three models based on deep learning-based visual language processing are fine-tuned using QLoRA.
  arXiv  Detail & Related papers  (2026-06-28T13:01:54Z)
• GLM-OCR Technical Report [65.42028025507491]
  GLM-OCR is an efficient compact model designed for real-world document understanding.<n>It combines a CogViT visual encoder with a GLM language decoder, achieving a strong balance between computational efficiency and recognition performance.<n>Extensive evaluations on public benchmarks and industrial scenarios show that GLM-OCR achieves competitive or state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-11T15:55:47Z)
• Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Models [2.1942030377331245]
  Vision-language models (VLMs) can read text from images, but where does this optical character recognition (OCR) information enter the language processing stream?<n>We investigate the OCR routing mechanism across three architecture families using causal interventions.
  arXiv  Detail & Related papers  (2026-02-26T12:06:02Z)
• LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR [0.29410438275861583]
  We present textbfLightOnOCR-2-1B, a multilingual vision--language model that converts document images into clean, naturally ordered text without brittle OCR pipelines.<n>Trained on a large-scale, high-quality distillation mix with strong coverage of scans, French documents, and scientific PDFs, LightOnOCR-2 achieves state-of-the-art results on OlmOCR-Bench.<n>We release model checkpoints under Apache 2.0, and publicly release the dataset and textbfLightOnOCR-bbox-bench evaluation under their respective licenses.
  arXiv  Detail & Related papers  (2026-01-20T18:58:32Z)
• PubMed-OCR: PMC Open Access OCR Annotations [0.4277619369763994]
  PubMed-OCR is an OCR-centric corpus of scientific articles derived from PubMed Central Open Access PDFs.<n>Each page image is annotated with Google Cloud Vision and released in a compact schema with word, line, and paragraph-level bounding boxes.<n>The corpus spans 209.5K articles (1.5M pages; 1.3B words) and supports layout-aware modeling, coordinate-grounded QA, and evaluation of OCR-dependent pipelines.
  arXiv  Detail & Related papers  (2026-01-16T16:44:50Z)
• MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patterns [80.05126590825121]
  MonkeyOCR v1.5 is a unified vision-language framework that enhances both layout understanding and content recognition.<n>To address complex table structures, we propose a visual consistency-based reinforcement learning scheme.<n>Two specialized modules, Image-Decoupled Table Parsing and Type-Guided Table Merging, are introduced to enable reliable parsing of tables.
  arXiv  Detail & Related papers  (2025-11-13T15:12:17Z)
• olmOCR 2: Unit Test Rewards for Document OCR [29.547676834557105]
  olmOCR 2 is the latest in our family of powerful OCR systems for converting digitized print documents, like PDFs, into clean, naturally ordered plain text.<n> olmOCR 2 is powered by olmOCR-2-7B-1025, a specialized, 7B vision language model (VLM) trained using reinforcement learning.<n>We show that RL training on these test cases results in state-of-the-art performance on olmOCR-Bench, our English-language OCR benchmark.
  arXiv  Detail & Related papers  (2025-10-22T17:53:02Z)
• ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links [57.514511353084565]
  We introduce a new domain-agnostic framework for selecting a best-performing approach and annotating cross-document links.<n>We apply our framework in two distinct domains -- peer review and news.<n>The resulting novel datasets lay foundation for numerous cross-document tasks like media framing and peer review.
  arXiv  Detail & Related papers  (2025-09-01T11:32:24Z)
• MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm [60.14048367611333]
  MonkeyOCR is a vision-language model for document parsing.<n>It advances the state of the art by leveraging a Structure-Recognition-Relation (SRR) triplet paradigm.
  arXiv  Detail & Related papers  (2025-06-05T16:34:57Z)
• Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing [46.14775667559124]
  layoutRL is an end-to-end reinforcement learning framework that trains models to be explicitly layout-aware.<n>We will publicly release our code and dataset to accelerate progress in robust document understanding.
  arXiv  Detail & Related papers  (2025-06-01T15:19:52Z)
• Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval [38.569818461453394]
  Retrieval-Augmented Generation (RAG) is a technique for grounding responses in external documents.<n>Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text.<n>Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR.
  arXiv  Detail & Related papers  (2025-05-08T21:54:02Z)
• LOCR: Location-Guided Transformer for Optical Character Recognition [55.195165959662795]
  We propose LOCR, a model that integrates location guiding into the transformer architecture during autoregression. We train the model on a dataset comprising over 77M text-location pairs from 125K academic document pages, including bounding boxes for words, tables and mathematical symbols. It outperforms all existing methods in our test set constructed from arXiv, as measured by edit distance, BLEU, METEOR and F-measure.
  arXiv  Detail & Related papers  (2024-03-04T15:34:12Z)
• Text Embeddings by Weakly-Supervised Contrastive Pre-training [98.31785569325402]
  E5 is a family of state-of-the-art text embeddings that transfer well to a wide range of tasks. E5 can be readily used as a general-purpose embedding model for any tasks requiring a single-vector representation of texts.
  arXiv  Detail & Related papers  (2022-12-07T09:25:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.