Vulnerability Analysis of Transformer-based Optical Character Recognition to Adversarial Attacks

• URL: http://arxiv.org/abs/2311.17128v1
• Date: Tue, 28 Nov 2023 15:22:23 GMT
• Title: Vulnerability Analysis of Transformer-based Optical Character Recognition to Adversarial Attacks
• Abstract summary: We develop and assess algorithms for both targeted and untargeted attacks. We find that TrOCR is highly vulnerable to untargeted attacks and somewhat less vulnerable to targeted attacks.
• Score: 1.8130068086063336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in Optical Character Recognition (OCR) have been driven by transformer-based models. OCR systems are critical in numerous high-stakes domains, yet their vulnerability to adversarial attack remains largely uncharted territory, raising concerns about security and compliance with emerging AI regulations. In this work we present a novel framework to assess the resilience of Transformer-based OCR (TrOCR) models. We develop and assess algorithms for both targeted and untargeted attacks. For the untargeted case, we measure the Character Error Rate (CER), while for the targeted case we use the success ratio. We find that TrOCR is highly vulnerable to untargeted attacks and somewhat less vulnerable to targeted attacks. On a benchmark handwriting data set, untargeted attacks can cause a CER of more than 1 without being noticeable to the eye. With a similar perturbation size, targeted attacks can lead to success rates of around $25\%$ -- here we attacked single tokens, requiring TrOCR to output the tenth most likely token from a large vocabulary.

Related papers

• Adversarial Attacks on Deep OCR Systems [14.333045298417828]
  We present the first pure black-box adversarial attack against a generative OCR vision-language model.<n>Pilot experiments on Deep-OCR validate the string-only attack and evaluation pipeline.<n>We avoid claiming targeted success until the pre-registered evaluation is complete.
  arXiv  Detail & Related papers  (2026-08-07T15:05:12Z)
• ICL-EVADER: Zero-Query Black-Box Evasion Attacks on In-Context Learning and Their Defenses [8.57098009274006]
  In-context learning (ICL) has become a powerful, data-efficient paradigm for text classification using large language models.<n>We introduce ICL-Evader, a novel black-box evasion attack framework that operates under a highly practical zero-query threat model.
  arXiv  Detail & Related papers  (2026-01-29T11:50:50Z)
• RLCracker: Exposing the Vulnerability of LLM Watermarks with Adaptive RL Attacks [18.75982610851903]
  We introduce adaptive robustness radius, a formal metric that quantifies watermark resilience against adaptive adversaries.<n>We propose RLCracker, a reinforcement learning (RL)-based adaptive attack that erases watermarks while preserving semantic fidelity.<n>Our results confirm that adaptive attacks are broadly effective and pose a fundamental threat to current watermarking defenses.
  arXiv  Detail & Related papers  (2025-09-25T09:08:02Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems [101.68501850486179]
  We explore adversarial attacks against retrieval-augmented generation (RAG) systems to identify their vulnerabilities.<n>This task aims to find imperceptible perturbations that retrieve a target document, originally excluded from the initial top-$k$ candidate set.<n>We propose ReGENT, a reinforcement learning-based framework that tracks interactions between the attacker and the target RAG.
  arXiv  Detail & Related papers  (2025-05-24T08:19:25Z)
• AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning [93.77763753231338]
  Adversarial Contrastive Prompt Tuning (ACPT) is proposed to fine-tune the CLIP image encoder to extract similar embeddings for any two intermediate adversarial queries. We show that ACPT can detect 7 state-of-the-art query-based attacks with $>99%$ detection rate within 5 shots. We also show that ACPT is robust to 3 types of adaptive attacks.
  arXiv  Detail & Related papers  (2024-08-04T09:53:50Z)
• Relaxing Graph Transformers for Adversarial Attacks [49.450581960551276]
  Graph Transformers (GTs) surpassed Message-Passing GNNs on several benchmarks, their adversarial robustness properties are unexplored. We overcome these challenges by targeting three representative architectures based on (1) random-walk PEs, (2) pair-wise-short-paths, and (3) spectral perturbations. Our evaluation reveals that they can be catastrophically fragile and underlines our work's importance and the necessity for adaptive attacks.
  arXiv  Detail & Related papers  (2024-07-16T14:24:58Z)
• Improving the Robustness of Object Detection and Classification AI models against Adversarial Patch Attacks [2.963101656293054]
  We analyze attack techniques and propose a robust defense approach. We successfully reduce model confidence by over 20% using adversarial patch attacks that exploit object shape, texture and position. Our inpainting defense approach significantly enhances model resilience, achieving high accuracy and reliable localization despite the adversarial attacks.
  arXiv  Detail & Related papers  (2024-03-04T13:32:48Z)
• Asymmetric Bias in Text-to-Image Generation with Adversarial Attacks [21.914674640285337]
  This paper focuses on analyzing factors associated with attack success rates (ASR) We introduce a new attack objective - entity swapping using adversarial suffixes and two gradient-based attack algorithms. We identify conditions that result in a success probability of 60% for adversarial attacks and others where this likelihood drops below 5%.
  arXiv  Detail & Related papers  (2023-12-22T05:10:32Z)
• PRAT: PRofiling Adversarial aTtacks [52.693011665938734]
  We introduce a novel problem of PRofiling Adversarial aTtacks (PRAT) Given an adversarial example, the objective of PRAT is to identify the attack used to generate it. We use AID to devise a novel framework for the PRAT objective.
  arXiv  Detail & Related papers  (2023-09-20T07:42:51Z)
• On the feasibility of attacking Thai LPR systems with adversarial examples [4.176752121302987]
  This work presents a feasibility study of performing adversarial attacks on an OCR system tailored specifically for the Thai language. We propose a new type of adversarial attack based on the emphsemi-targeted scenario and show that this scenario is highly realistic in LPR applications.
  arXiv  Detail & Related papers  (2023-01-13T12:17:01Z)
• On Trace of PGD-Like Adversarial Attacks [77.75152218980605]
  Adversarial attacks pose safety and security concerns for deep learning applications. We construct Adrial Response Characteristics (ARC) features to reflect the model's gradient consistency. Our method is intuitive, light-weighted, non-intrusive, and data-undemanding.
  arXiv  Detail & Related papers  (2022-05-19T14:26:50Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)
• Detecting Adversarial Examples for Speech Recognition via Uncertainty Quantification [21.582072216282725]
  Machine learning systems and, specifically, automatic speech recognition (ASR) systems are vulnerable to adversarial attacks. In this paper, we focus on hybrid ASR systems and compare four acoustic models regarding their ability to indicate uncertainty under attack. We are able to detect adversarial examples with an area under the receiving operator curve score of more than 0.99.
  arXiv  Detail & Related papers  (2020-05-24T19:31:02Z)
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks [65.20660287833537]
  In this paper we propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness.
  arXiv  Detail & Related papers  (2020-03-03T18:15:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.