Beyond OCR Accuracy: Text-Centric VQA Under Image Degradation with Modular and End-to-End

• URL: http://arxiv.org/abs/2609.13815v1
• Date: Sat, 12 Sep 2026 08:57:29 GMT
• Title: Beyond OCR Accuracy: Text-Centric VQA Under Image Degradation with Modular and End-to-End
• Abstract summary: Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images.<n>While modular OCR-based pipelines and end-to-end vision-language models are both widely used for this task, their comparative robustness under degraded conditions remains underexplored.<n>We present an empirical study comparing two modular pipelines with SA-DBNet, a custom detector architecture combining ResNet-18 with self-attention spatial modeling and deformable convolutions.
• Score: 5.790885750156007
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images, a task made substantially harder when images suffer from real-world degradation such as motion blur, low resolution, or compression artifacts. While modular OCR-based pipelines and end-to-end vision-language models are both widely used for this task, their comparative robustness under degraded conditions remains underexplored. We present an empirical study comparing two modular pipelines with SA-DBNet, a custom detector architecture combining ResNet-18 with self-attention spatial modeling and deformable convolutions against an end-to-end vision-language baseline, evaluated on 4013 degraded images with 7000 question-answer pairs. Fine-tuned modular pipelines achieve up to 57.50% exact-match accuracy versus 38.00% for the end-to-end baseline, with domain-specific fine-tuning yielding a gain of up to 29.50 percentage points. Critically, we find that conventional OCR error metrics like Character Error Rate and Word Error Rate are unreliable predictors of downstream VQA performance, as semantic reasoning can compensate for recognition failures when contextual cues are present. These findings highlight the importance of task-aware evaluation for text-centric VQA systems under realistic visual conditions. Codes are available here

Related papers

• Human-Grounded Calibration for Long-Text Image-Text Congruence in Vision-Language Models [46.28810845443601]
  Long-text image--text congruence scoring is increasingly important for vision-language systems.<n>This paper proposes Congruency Score (CS), a lightweight calibration layer that maps image--text similarity evidence into a bounded score.
  arXiv  Detail & Related papers  (2026-09-14T14:26:53Z)
• Beyond Blind Compliance: Benchmarking Task Verification in OCR Reasoning [23.222077284906618]
  Existing evaluations assume that every task is valid and answerable, but in real-world OCR scenarios this assumption often fails.<n>We introduce VeriOCRBench, a benchmark built from source images drawn from 8 OCR-related datasets and spanning 8 real-world image domains.<n>Built with a Visual Atomic Fact (VAF)-anchored pipeline and full human auditing, VeriOCRBench enables decoupled evaluation of task verification, root-cause diagnosis, and over-refusal.
  arXiv  Detail & Related papers  (2026-08-31T18:39:46Z)
• Visual Distortion Detection in UGC Images Using Large Multimodal Models [40.6196464667839]
  We propose textbftextitVIGIL, which leverages the large language model (LLM) architecture for precise visual distortion detection.<n>Our model consistently outperforms strong baselines on both in-domain synthetic distortion detection and textitS2A tasks.
  arXiv  Detail & Related papers  (2026-08-10T05:00:12Z)
• When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning [73.56343820592399]
  We introduce Visualized Task Semantics (VTS), a controlled intervention that moves the question into the image while keeping the source problem and answer fixed.<n>Across six MLLMs and four benchmarks, accuracy drops in all 24 model-task pairs, by 17.8 points on average.<n>We present prompt-region grounding, whose core design aligns the question region with typed semantics and recovers its clean representation from a masked view.
  arXiv  Detail & Related papers  (2026-08-05T11:46:15Z)
• Decoupling semantics from vision: A framework for faithful visual-text compression evaluation [5.942515516844501]
  We introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality.<n>By eliminating textual dependencies, our benchmark guarantees that the evaluation results are purely reflective of VTC quality.
  arXiv  Detail & Related papers  (2026-08-03T07:56:43Z)
• Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback [51.08692072066352]
  Text-to-image (T2I) models still exhibit localized, subtle, and structurally complex failures.<n>Structured Defect Grounding casts T2I diagnosis as structured set prediction by modeling each defect as a location, type, reason, importance.<n>Our detector outperforms leading proprietary VLMs on structured defect grounding.
  arXiv  Detail & Related papers  (2026-06-04T13:03:16Z)
• Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs [23.958966900531692]
  Multimodal large language models (MLLMs) can process text presented as images, yet they often perform worse than when the same content is provided as textual tokens.<n>We systematically diagnose this "modality gap" by evaluating seven MLLMs across seven benchmarks in five input modes.
  arXiv  Detail & Related papers  (2026-03-10T02:14:23Z)
• Interleaved Scene Graphs for Interleaved Text-and-Image Generation Assessment [53.45813302866466]
  We present ISG, a comprehensive evaluation framework for interleaved text-and-image generation.<n>ISG evaluates responses on four levels of granularity: holistic, structural, block-level, and image-specific.<n>In conjunction with ISG, we introduce a benchmark, ISG-Bench, encompassing 1,150 samples across 8 categories and 21 subcategories.
  arXiv  Detail & Related papers  (2024-11-26T07:55:57Z)
• UATVR: Uncertainty-Adaptive Text-Video Retrieval [90.8952122146241]
  A common practice is to transfer text-video pairs to the same embedding space and craft cross-modal interactions with certain entities. We propose an Uncertainty-language Text-Video Retrieval approach, termed UATVR, which models each look-up as a distribution matching procedure.
  arXiv  Detail & Related papers  (2023-01-16T08:43:17Z)
• Learning to Decompose Visual Features with Latent Textual Prompts [140.2117637223449]
  We propose Decomposed Feature Prompting (DeFo) to improve vision-language models. Our empirical study shows DeFo's significance in improving the vision-language models.
  arXiv  Detail & Related papers  (2022-10-09T15:40:13Z)
• Towards Escaping from Language Bias and OCR Error: Semantics-Centered Text Visual Question Answering [14.010472385359163]
  Texts in scene images convey critical information for scene understanding and reasoning. Current TextVQA models do not center on the text and suffer from several limitations. We propose a novel Semantics-Centered Network (SC-Net) that consists of an instance-level contrastive semantic prediction module and a semantics-centered transformer module.
  arXiv  Detail & Related papers  (2022-03-24T08:21:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.