DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth

• URL: http://arxiv.org/abs/2607.16203v1
• Date: Tue, 05 May 2026 10:59:34 GMT
• Title: DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth
• Abstract summary: We conduct a systematic evaluation of text recognition performance across a diverse set of OCR engines and state-of-the-art MLLMs.<n>Motivated by the limited contextual reasoning capabilities of many OCR engines and the high cost of manual annotations, we propose DocOCR-Eval.
• Score: 24.056193540937144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Document parsing is a foundational step for document understanding tasks such as visual question answering and key information extraction, as it transforms unstructured scanned images into structured representations by extracting textual, visual, and layout information. While numerous Optical Character Recognition (OCR) engines and multimodal large language models (MLLMs) have been developed for this purpose, selecting an appropriate document parsing solution for a given document collection remains challenging, particularly in label-scarce settings. In this work, we conduct a systematic evaluation of text recognition performance across a diverse set of OCR engines and state-of-the-art MLLMs on multiple scanned document benchmarks spanning different domains and languages. Motivated by the limited contextual reasoning capabilities of many OCR engines and the high cost of manual annotations, we propose DocOCR-Eval, an annotation-free evaluation framework for automatic OCR assessment and selection. DocOCR-Eval employs a three-staged correction and ranking strategy to approximate annotation-based tool ordering without ground-truth labels. We show that aggregating across multiple MLLMs progressively improves alignment with annotation-based rankings. Extensive experiments further demonstrate that reliable OCR tool selection can be achieved in realistic, label-limited settings, providing practical guidance for deploying document parsing systems across diverse real-world document collections.

Related papers

• NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Documents [18.60763757268658]
  We propose NaviDC-OCR, a unified framework for document parsing.<n> NaviDC-OCR introduces deformation-aware learning to incorporate geometric perception into VLMs.<n>It achieves state-of-the-art performance across diverse document parsing benchmarks.
  arXiv  Detail & Related papers  (2026-08-13T07:34:21Z)
• MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing [74.84107522458798]
  MPDocBench-Parse is a benchmark for multi-page document parsing in real-world applications.<n>It contains 433 manually annotated documents with 3,246 pages, covering 15 document types in English and Chinese, with diverse layout styles.
  arXiv  Detail & Related papers  (2026-05-21T07:36:41Z)
• Multimodal OCR: Parse Anything from Documents [72.69545534962234]
  dots.mocr treats visual elements such as charts, diagrams, tables, and icons as first-class parsing targets.<n>It reconstructs both text and graphics as structured outputs, enabling more faithful document reconstruction.<n>It supports end-to-end training over heterogeneous document elements.
  arXiv  Detail & Related papers  (2026-03-13T14:42:21Z)
• DISCO: Document Intelligence Suite for COmparative Evaluation [1.4425299138308667]
  Document intelligence requires accurate text extraction and reliable reasoning over document content.<n>We evaluate optical character recognition (OCR) pipelines and vision-language models (VLMs) separately on parsing and question answering across diverse document types.
  arXiv  Detail & Related papers  (2026-03-04T14:47:34Z)
• MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering [51.19392014547221]
  Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures.<n>Current approaches rely on query-agnostic document representations that overlook salient content.<n>We propose the Multimodal Adaptive Retrieval-Augmented (MARA) framework, which introduces query-adaptive generation.
  arXiv  Detail & Related papers  (2026-02-01T12:27:40Z)
• Digitization of Document and Information Extraction using OCR [0.0]
  This document presents a framework for text extraction that merges Optical Character Recognition (OCR) techniques with Large Language Models (LLMs)<n>Scanned files are processed using OCR engines, while digital files are interpreted through layout-aware libraries.<n>The extracted raw text is then analyzed by an LLM to identify key-value pairs and resolve ambiguities.
  arXiv  Detail & Related papers  (2025-06-11T16:03:01Z)
• Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval [38.569818461453394]
  Retrieval-Augmented Generation (RAG) is a technique for grounding responses in external documents.<n>Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text.<n>Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR.
  arXiv  Detail & Related papers  (2025-05-08T21:54:02Z)
• VISTA-OCR: Towards generative and interactive end to end OCR models [3.7548609506798494]
  VISTA-OCR is a lightweight architecture that unifies text detection and recognition within a single generative model.<n>Built on an encoder-decoder architecture, VISTA-OCR is progressively trained, starting with the visual feature extraction phase.<n>To enhance the model's capabilities, we built a new dataset composed of real-world examples enriched with bounding box annotations and synthetic samples.
  arXiv  Detail & Related papers  (2025-04-04T17:39:53Z)
• QID: Efficient Query-Informed ViTs in Data-Scarce Regimes for OCR-free Visual Document Understanding [53.69841526266547]
  Fine-tuning a pre-trained Vision-Language Model with new datasets often falls short in optimizing the vision encoder.<n>We introduce QID, a novel, streamlined, architecture-preserving approach that integrates query embeddings into the vision encoder.
  arXiv  Detail & Related papers  (2025-04-03T18:47:16Z)
• Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking [58.69615583599489]
  Deliberate Thinking based Retriever (Debater) is a novel approach that enhances document representations by incorporating a step-by-step thinking process.<n>Debater significantly outperforms existing methods across several retrieval benchmarks.
  arXiv  Detail & Related papers  (2025-02-18T15:56:34Z)
• mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding [55.4806974284156]
  Document understanding refers to automatically extract, analyze and comprehend information from digital documents, such as a web page. Existing Multi-model Large Language Models (MLLMs) have demonstrated promising zero-shot capabilities in shallow OCR-free text recognition.
  arXiv  Detail & Related papers  (2023-07-04T11:28:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.