How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations

• URL: http://arxiv.org/abs/2606.26041v1
• Date: Wed, 24 Jun 2026 17:15:42 GMT
• Title: How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations
• Abstract summary: Vision-language models (VLMs) have achieved strong performance on OCR-based benchmarks, but their robustness under controlled visual degradation remains insufficiently understood.<n>We introduce OCR-Robust, a benchmark designed for evaluating OCR reasoning robustness under visual perturbations.<n>We conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross-model discriminability.
• Score: 17.186250064889887
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) have achieved strong performance on OCR-based benchmarks and increasingly focused on text-rich understanding, but their robustness under controlled visual degradation remains insufficiently understood. This gap is critical for OCR reasoning, where visual corruption can induce OCR errors and structural distortions, thereby introducing uncertainty into the reasoning task. To systematically study this problem, we introduce OCR-Robust, a benchmark designed for evaluating OCR reasoning robustness under visual perturbations. It contains 812 samples across two complementary subsets: OCR1.0, covering documents, scene text, receipts, handwriting, and mathematical content, and OCR2.0, focusing on charts, geometry diagrams, and tables. To enable efficient yet informative evaluation, we conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross-model discriminability. We evaluate robustness using clean accuracy, Relative Corruption Retention (RCR), Worst-Case Retention (WCR), and a composite Corruption Robustness Index (CRI), and benchmark 18 models spanning proprietary systems, open-source VLMs, and OCR+LLM pipelines. Our results show that higher clean accuracy does not necessarily imply stronger robustness, and that models can suffer pronounced degradation in the worst case on OCR tasks that are sensitive to structure, and charts and tables are substantially more fragile than document-like inputs under perturbation.

Related papers

• OCR-EDR: Rendering-Aware Diagnosis and Repair for Closed-Loop OCR Improvement [14.643765225504078]
  We propose OCR-EDR (OCR Error Diagnosis and Repair), a rendering-aware framework that advances from fine-grained diagnosis to iterative repair.<n>We construct OCRErrBench from diverse real OCR predictions, covering text and formulas, exact and rendering-equivalent positives, and genuine errors.<n> OCR-EDR turns fine-grained OCR analysis into verified corrections and performance gains.
  arXiv  Detail & Related papers  (2026-09-03T07:02:17Z)
• ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation [29.744176697656147]
  We formulate adversarial OCR as a textbfgrounded OCR perception task and introduce textbfAdvSpot, the first benchmark for grounded adversarial OCR evaluation.<n>ArmorOCR is a two-stage training framework for robust adversarial OCR perception.<n>Experiments on our AdvSpot, other adversarial OCR benchmarks, and general OCR benchmarks demonstrate that ArmorOCR consistently improves adversarial OCR perception.
  arXiv  Detail & Related papers  (2026-08-20T14:52:06Z)
• Cross-Resolution Semantic Transfer for Robust Text-to-Image Retrieval in Low-Resolution Surveillance [62.560024755525546]
  Cross-Resolution Semantic Transfer (CRST) is a CLIP-style framework with three modules: resolution-conditioned reasoning, text-guided refinement and CR-RDA.<n> Experiments show that CRST improves ultra-low-resolution Rank-1 and mAP on average by 5.7% and 5.3%, while stabilizing mixed-resolution retrieval without sacrificing high-resolution accuracy.
  arXiv  Detail & Related papers  (2026-06-29T15:24:03Z)
• When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation [2.8437065210971872]
  Industrial Retrieval-Augmented Generation (RAG) systems depend on optical character recognition (OCR) to transform visual documents into text.<n>Existing OCR benchmarks rely on character-level metrics, which inadequately measure downstream RAG effectiveness under real-world conditions.<n>We introduce an OCR benchmark for industrial RAG systems covering 11 challenging document types.
  arXiv  Detail & Related papers  (2026-04-29T09:42:42Z)
• From Plausibility to Verifiability: Risk-Controlled Generative OCR for Vision-Language Models [13.2889725132666]
  Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures.<n>We identify a core deployment misalignment in generative OCR and propose a model-agnostic Geometric Risk Controller.<n> Experiments on frozen VLM backbones and standard OCR benchmarks show consistent reductions in extreme-error risk and catastrophic over-generation at predictable coverage costs.
  arXiv  Detail & Related papers  (2026-03-20T09:28:09Z)
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding [54.05243949024302]
  Existing robust MLLMs rely on implicit training/adaptation that focuses solely on visual encoder generalization.<n>We propose Robust-R1, a novel framework that explicitly models visual degradations through structured reasoning chains.<n>Our approach integrates: (i) supervised fine-tuning for degradation-aware reasoning foundations, (ii) reward-driven alignment for accurately perceiving degradation parameters, and (iii) dynamic reasoning depth scaling adapted to degradation intensity.
  arXiv  Detail & Related papers  (2025-12-19T12:56:17Z)
• Reasoning-OCR: Can Large Multimodal Models Solve Complex Logical Reasoning Problems from OCR Cues? [73.35232225256968]
  Reasoning-OCR challenges LMMs to solve complex reasoning problems based on the cues that can be extracted from rich visual-text.<n>Our evaluation offers some insights for proprietary and open-source LMMs in different reasoning challenges.
  arXiv  Detail & Related papers  (2025-05-19T06:45:18Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation [39.83221375597683]
  Retrieval-augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge to reduce hallucinations and incorporate up-to-date information without retraining.<n>As an essential part of RAG, external knowledge bases are commonly built by extracting structured data from unstructured PDF documents using Optical Character Recognition (OCR)<n>In this paper, we introduce OHRBench, the first benchmark for understanding the cascading impact of OCR on RAG systems.
  arXiv  Detail & Related papers  (2024-12-03T17:23:47Z)
• Cross-modal Active Complementary Learning with Self-refining Correspondence [54.61307946222386]
  We propose a Cross-modal Robust Complementary Learning framework (CRCL) to improve the robustness of existing methods. ACL exploits active and complementary learning losses to reduce the risk of providing erroneous supervision. SCC utilizes multiple self-refining processes with momentum correction to enlarge the receptive field for correcting correspondences.
  arXiv  Detail & Related papers  (2023-10-26T15:15:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.