From Plausibility to Verifiability: Risk-Controlled Generative OCR for Vision-Language Models

• URL: http://arxiv.org/abs/2603.19790v1
• Date: Fri, 20 Mar 2026 09:28:09 GMT
• Title: From Plausibility to Verifiability: Risk-Controlled Generative OCR for Vision-Language Models
• Abstract summary: Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures.<n>We identify a core deployment misalignment in generative OCR and propose a model-agnostic Geometric Risk Controller.<n> Experiments on frozen VLM backbones and standard OCR benchmarks show consistent reductions in extreme-error risk and catastrophic over-generation at predictable coverage costs.
• Score: 13.2889725132666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures. We identify a core deployment misalignment in generative OCR. Autoregressive decoding favors semantic plausibility, whereas OCR requires outputs that are visually grounded and geometrically verifiable. This mismatch produces severe errors, especially over-generation and unsupported substitutions, creating deployment risk even when benchmark accuracy remains high. We therefore formulate frozen VLM OCR as a selective accept/abstain problem and propose a model-agnostic Geometric Risk Controller. The controller probes multiple structured views of the same input, applies lightweight structural screening, and accepts a transcription only when cross-view consensus and stability satisfy predefined criteria, yielding a small family of operating points. Experiments on frozen VLM backbones and standard OCR benchmarks show consistent reductions in extreme-error risk and catastrophic over-generation at predictable coverage costs. Reliable deployment of generative OCR with frozen VLMs benefits from explicit system-level risk control rather than unconstrained generation.

Related papers

• Adversarial Attacks on Deep OCR Systems [14.333045298417828]
  We present the first pure black-box adversarial attack against a generative OCR vision-language model.<n>Pilot experiments on Deep-OCR validate the string-only attack and evaluation pipeline.<n>We avoid claiming targeted success until the pre-registered evaluation is complete.
  arXiv  Detail & Related papers  (2026-08-07T15:05:12Z)
• When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents [6.6457932580691015]
  Vision-Language Models (VLMs) have emerged as strong alternatives to traditional OCR systems.<n>In this work, we benchmark traditional OCR systems and VLM-based approaches on the Berrutti dataset.
  arXiv  Detail & Related papers  (2026-07-27T07:19:00Z)
• How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations [17.186250064889887]
  Vision-language models (VLMs) have achieved strong performance on OCR-based benchmarks, but their robustness under controlled visual degradation remains insufficiently understood.<n>We introduce OCR-Robust, a benchmark designed for evaluating OCR reasoning robustness under visual perturbations.<n>We conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross-model discriminability.
  arXiv  Detail & Related papers  (2026-06-24T17:15:42Z)
• PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution [51.96078493242164]
  PRISM is a single-step diffusion-based Text-SR framework.<n> PRISM achieves state-of-the-art performance with millisecond-level inference.
  arXiv  Detail & Related papers  (2026-05-13T05:31:06Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation [2.8437065210971872]
  Industrial Retrieval-Augmented Generation (RAG) systems depend on optical character recognition (OCR) to transform visual documents into text.<n>Existing OCR benchmarks rely on character-level metrics, which inadequately measure downstream RAG effectiveness under real-world conditions.<n>We introduce an OCR benchmark for industrial RAG systems covering 11 challenging document types.
  arXiv  Detail & Related papers  (2026-04-29T09:42:42Z)
• Causal Bootstrapped Alignment for Unsupervised Video-Based Visible-Infrared Person Re-Identification [52.784239635604735]
  VVI-ReID is a critical technique for all-day surveillance, where temporal information provides additional cues beyond static images.<n>Existing approaches rely heavily on fully supervised learning with expensive cross-modality annotations, limiting scalability.<n>We propose a Causal Bootstrapped Alignment framework that explicitly exploits inherent video priors.
  arXiv  Detail & Related papers  (2026-04-17T02:15:59Z)
• TopoReformer: Mitigating Adversarial Attacks Using Topological Purification in OCR Models [2.8272661103123258]
  Adversarially perturbed images of text can cause OCR systems to produce misleading or incorrect transcriptions from seemingly invisible changes to humans.<n>Existing defenses, such as adversarial training, input preprocessing, or post-recognition correction, are often model-specific, computationally expensive, and affect performance on unperturbed inputs.<n>TopoReformer is introduced, a model-agnostic reformation pipeline that mitigates adversarial perturbations while preserving the structural integrity of text images.
  arXiv  Detail & Related papers  (2025-11-19T19:01:51Z)
• Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal [31.458406135473805]
  We present UniCR, a unified framework that turns heterogeneous uncertainty evidence into a calibrated probability of correctness.<n>UniCR learns a lightweight calibration head with temperature scaling and proper scoring.<n>Experiments on short-form QA, code generation with execution tests, and retrieval-augmented long-form QA show consistent improvements in calibration metrics.
  arXiv  Detail & Related papers  (2025-09-01T13:14:58Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• Conditional Conformal Risk Adaptation [9.559062601251464]
  We develop a new score function for creating adaptive prediction sets that significantly improve conditional risk control for segmentation tasks.<n>We introduce a specialized probability calibration framework that enhances the reliability of pixel-wise inclusion estimates.<n>Our experiments on polyp segmentation demonstrate that all three methods provide valid marginal risk control and deliver more consistent conditional risk control.
  arXiv  Detail & Related papers  (2025-04-10T10:01:06Z)
• OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation [39.83221375597683]
  Retrieval-augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge to reduce hallucinations and incorporate up-to-date information without retraining.<n>As an essential part of RAG, external knowledge bases are commonly built by extracting structured data from unstructured PDF documents using Optical Character Recognition (OCR)<n>In this paper, we introduce OHRBench, the first benchmark for understanding the cascading impact of OCR on RAG systems.
  arXiv  Detail & Related papers  (2024-12-03T17:23:47Z)
• C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models [57.10361282229501]
  We propose C-RAG, the first framework to certify generation risks for RAG models. Specifically, we provide conformal risk analysis for RAG models and certify an upper confidence bound of generation risks. We prove that RAG achieves a lower conformal generation risk than that of a single LLM when the quality of the retrieval model and transformer is non-trivial.
  arXiv  Detail & Related papers  (2024-02-05T16:46:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.