ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation

• URL: http://arxiv.org/abs/2608.20122v1
• Date: Thu, 20 Aug 2026 14:52:06 GMT
• Title: ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation
• Abstract summary: We formulate adversarial OCR as a textbfgrounded OCR perception task and introduce textbfAdvSpot, the first benchmark for grounded adversarial OCR evaluation.<n>ArmorOCR is a two-stage training framework for robust adversarial OCR perception.<n>Experiments on our AdvSpot, other adversarial OCR benchmarks, and general OCR benchmarks demonstrate that ArmorOCR consistently improves adversarial OCR perception.
• Score: 29.744176697656147
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large multimodal models (LMMs) have demonstrated strong OCR recognition capabilities, yet remain vulnerable to adversarial visual text that is readable to humans but challenging for models to localize and recognize. Existing OCR benchmarks mainly focus on natural or document-style text, while adversarial OCR evaluations remain limited in scale, task coverage, or region-aware evaluation. In this paper, we formulate adversarial OCR as a \textbf{grounded OCR perception} task and introduce \textbf{AdvSpot}, the first benchmark for grounded adversarial OCR evaluation. AdvSpot comprises 390 images with region-level annotations, spanning 5 primary categories and 13 fine-grained adversarial OCR types. To address this challenge, we propose \textbf{ArmorOCR}, a two-stage training framework for robust adversarial OCR perception. ArmorOCR first acquires missing adversarial OCR perception from privileged transformed observations through On-Policy Self-Distillation (OPSD), and then refines grounded OCR perception through Group Relative Policy Optimization (GRPO) with task-conditioned rewards for localization, recognition, full spotting, and visual question answering (VQA). Experiments on our AdvSpot, other adversarial OCR benchmarks, and general OCR benchmarks demonstrate that ArmorOCR consistently improves adversarial OCR perception while preserving competitive general OCR capability.

Related papers

• Cross-Resolution Semantic Transfer for Robust Text-to-Image Retrieval in Low-Resolution Surveillance [62.560024755525546]
  Cross-Resolution Semantic Transfer (CRST) is a CLIP-style framework with three modules: resolution-conditioned reasoning, text-guided refinement and CR-RDA.<n> Experiments show that CRST improves ultra-low-resolution Rank-1 and mAP on average by 5.7% and 5.3%, while stabilizing mixed-resolution retrieval without sacrificing high-resolution accuracy.
  arXiv  Detail & Related papers  (2026-06-29T15:24:03Z)
• How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations [17.186250064889887]
  Vision-language models (VLMs) have achieved strong performance on OCR-based benchmarks, but their robustness under controlled visual degradation remains insufficiently understood.<n>We introduce OCR-Robust, a benchmark designed for evaluating OCR reasoning robustness under visual perturbations.<n>We conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross-model discriminability.
  arXiv  Detail & Related papers  (2026-06-24T17:15:42Z)
• When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation [2.8437065210971872]
  Industrial Retrieval-Augmented Generation (RAG) systems depend on optical character recognition (OCR) to transform visual documents into text.<n>Existing OCR benchmarks rely on character-level metrics, which inadequately measure downstream RAG effectiveness under real-world conditions.<n>We introduce an OCR benchmark for industrial RAG systems covering 11 challenging document types.
  arXiv  Detail & Related papers  (2026-04-29T09:42:42Z)
• TextShield-R1: Reinforced Reasoning for Tampered Text Detection [48.66129052598456]
  TextShield-R1 is the first reinforcement learning based MLLM solution for tampered text detection and reasoning.<n>We introduce Forensic Continual Pre-training, an easy-to-hard curriculum that well prepares the MLLM for tampered text detection.<n>We also introduce the Text Forensics Reasoning (TFR) benchmark, comprising over 45k real and tampered images across 16 languages.
  arXiv  Detail & Related papers  (2026-02-23T13:26:18Z)
• Reasoning-OCR: Can Large Multimodal Models Solve Complex Logical Reasoning Problems from OCR Cues? [73.35232225256968]
  Reasoning-OCR challenges LMMs to solve complex reasoning problems based on the cues that can be extracted from rich visual-text.<n>Our evaluation offers some insights for proprietary and open-source LMMs in different reasoning challenges.
  arXiv  Detail & Related papers  (2025-05-19T06:45:18Z)
• Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval [38.569818461453394]
  Retrieval-Augmented Generation (RAG) is a technique for grounding responses in external documents.<n>Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text.<n>Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR.
  arXiv  Detail & Related papers  (2025-05-08T21:54:02Z)
• Ocean-OCR: Towards General OCR Application via a Vision-Language Model [6.70908296002235]
  We present textbfOcean-OCR, a 3B MLLM with state-of-the-art performance on various OCR scenarios and comparable understanding ability on general tasks.<n>We demonstrate the superiority of Ocean-OCR through comprehensive experiments on open-source OCR benchmarks and across various OCR scenarios.
  arXiv  Detail & Related papers  (2025-01-26T15:20:39Z)
• CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy [50.78228433498211]
  CC-OCR comprises four OCR-centric tracks: multi-scene text reading, multilingual text reading, document parsing, and key information extraction.<n>It includes 39 subsets with 7,058 full annotated images, of which 41% are sourced from real applications, and released for the first time.<n>We evaluate nine prominent LMMs and reveal both the strengths and weaknesses of these models, particularly in text grounding, multi-orientation, and hallucination of repetition.
  arXiv  Detail & Related papers  (2024-12-03T07:03:25Z)
• Adversarial Training with OCR Modality Perturbation for Scene-Text Visual Question Answering [8.382903851560595]
  Scene-Text Visual Question Answering (ST-VQA) aims to understand scene text in images and answer questions related to the text content. Most existing methods heavily rely on the accuracy of Optical Character Recognition (OCR) systems. We propose a multimodal adversarial training architecture with spatial awareness capabilities.
  arXiv  Detail & Related papers  (2024-03-14T11:22:06Z)
• Cross-modal Active Complementary Learning with Self-refining Correspondence [54.61307946222386]
  We propose a Cross-modal Robust Complementary Learning framework (CRCL) to improve the robustness of existing methods. ACL exploits active and complementary learning losses to reduce the risk of providing erroneous supervision. SCC utilizes multiple self-refining processes with momentum correction to enlarge the receptive field for correcting correspondences.
  arXiv  Detail & Related papers  (2023-10-26T15:15:11Z)
• User-Centric Evaluation of OCR Systems for Kwak'wala [92.73847703011353]
  We show that utilizing OCR reduces the time spent in the manual transcription of culturally valuable documents by over 50%. Our results demonstrate the potential benefits that OCR tools can have on downstream language documentation and revitalization efforts.
  arXiv  Detail & Related papers  (2023-02-26T21:41:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.