Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformers

• URL: http://arxiv.org/abs/2608.06674v1
• Date: Fri, 07 Aug 2026 00:42:25 GMT
• Title: Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformers
• Abstract summary: Adversarial vulnerabilities remain a major concern for the safe deployment of neural networks.<n>Most existing attacks target the detection output rather than the attention mechanism that makes these models distinctive.<n>We introduce the first attack that directly optimize an encoder-attention objective under an imperceptible, bounded $ell_infty$ perturbation.
• Score: 35.58130964667448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial vulnerabilities remain a major concern for the safe deployment of neural networks, particularly in object detection, a core task embedded in many safety-critical systems. Detection transformers have emerged as leading object detectors, yet their adversarial robustness remains comparatively underexplored. Most existing attacks target the detection output rather than the attention mechanism that makes these models distinctive. In this paper, we introduce the first attack that directly optimizes an encoder-attention objective under an imperceptible, bounded $\ell_\infty$ perturbation. Rather than introducing an attacker-owned sink token through a visible patch, it drives the model's own attention toward a corrupted target. We argue that encoder attention concentrates the model's spatial reasoning, so corrupting it propagates through the detection pipeline more disruptively than perturbing the detection output alone. Our attack reduces DETR-R50 mAP on COCO from 42.1 to 0.97, a $\sim 4\times$ reduction in resulting mAP over the strongest existing attack under an identical perturbation budget and iteration count. We further show that this vulnerability is not specific to a particular corruption objective: across four qualitatively distinct targets, dispersion, re-ranking, permutation, and peak-suppression, detection consistently drops below 3 mAP, suggesting that the weakness arises from disrupting the attention structure itself rather than from any single target. Finally, we demonstrate that the attack generalizes across attention formulations, reducing DINO-Swin-L from 56.8 to 1.44 mAP against 7.3 for the strongest prior attack, establishing state-of-the-art on both dense and deformable attention.

Related papers

• Backdoor Mitigation in Object Detection via Adversarial Fine-Tuning [10.393154496941527]
  Backdoor attacks pose a serious threat to safety-critical vision systems.<n>Adrial fine-tuning is a common backdoor mitigation approach in classification.<n>We present a detection-aware adversarial fine-tuning framework for mitigating object-detection backdoors.
  arXiv  Detail & Related papers  (2026-05-07T09:33:06Z)
• Fight Poison with Poison: Enhancing Robustness in Few-shot Machine-Generated Text Detection with Adversarial Training [3.427574505782467]
  We propose RAG-GuidEd Attacker Strengthens ConTrastive Few-shot Detector (REACT)<n>REACT couples a humanization-oriented attacker with a target detector.<n>We show that REACT improves average detection F1 by 4.95 points over 8 state-of-the-art detectors.
  arXiv  Detail & Related papers  (2026-05-04T09:16:57Z)
• Budgeted Adversarial Attack against Graph-Based Anomaly Detection in Sensor Networks [1.2891210250935148]
  Graph Neural Networks (GNNs) have emerged as powerful models for anomaly detection in sensor networks.<n>We introduce BETA, a novel grey-box evasion attack targeting such GNN-based detectors.<n>We show that BETA reduces the detection accuracy of state-of-the-art GNN-based detectors by 30.62 to 39.16% on average.
  arXiv  Detail & Related papers  (2025-09-22T16:30:19Z)
• Adversarial Attention Perturbations for Large Object Detection Transformers [6.845910470068847]
  Adversarial perturbations are useful tools for exposing vulnerabilities in neural networks.<n>This paper presents an Attention-Focused Offensive Gradient (AFOG) attack against object detection transformers.
  arXiv  Detail & Related papers  (2025-08-05T01:31:10Z)
• AnywhereDoor: Multi-Target Backdoor Attacks on Object Detection [9.539021752700823]
  AnywhereDoor is a multi-target backdoor attack for object detection.<n>It allows adversaries to make objects disappear, fabricate new ones or mislabel them, either across all object classes or specific ones.<n>It improves attack success rates by 26% compared to adaptations of existing methods for such flexible control.
  arXiv  Detail & Related papers  (2025-03-09T09:24:24Z)
• Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement [68.31147013783387]
  We observe that the attention mechanism is vulnerable to patch-based adversarial attacks. In this paper, we propose a Robust Attention Mechanism (RAM) to improve the robustness of the semantic segmentation model.
  arXiv  Detail & Related papers  (2024-01-03T13:58:35Z)
• Enhancing Infrared Small Target Detection Robustness with Bi-Level Adversarial Framework [61.34862133870934]
  We propose a bi-level adversarial framework to promote the robustness of detection in the presence of distinct corruptions. Our scheme remarkably improves 21.96% IOU across a wide array of corruptions and notably promotes 4.97% IOU on the general benchmark.
  arXiv  Detail & Related papers  (2023-09-03T06:35:07Z)
• Threatening Patch Attacks on Object Detection in Optical Remote Sensing Images [55.09446477517365]
  Advanced Patch Attacks (PAs) on object detection in natural images have pointed out the great safety vulnerability in methods based on deep neural networks. We propose a more Threatening PA without the scarification of the visual quality, dubbed TPA. To the best of our knowledge, this is the first attempt to study the PAs on object detection in O-RSIs, and we hope this work can get our readers interested in studying this topic.
  arXiv  Detail & Related papers  (2023-02-13T02:35:49Z)
• Object-fabrication Targeted Attack for Object Detection [54.10697546734503]
  adversarial attack for object detection contains targeted attack and untargeted attack. New object-fabrication targeted attack mode can mislead detectors tofabricate extra false objects with specific target labels.
  arXiv  Detail & Related papers  (2022-12-13T08:42:39Z)
• Adversarial Attack and Defense of YOLO Detectors in Autonomous Driving Scenarios [3.236217153362305]
  We present an effective attack strategy aiming the objectness aspect of visual detection in autonomous vehicles. Experiments show that the proposed attack targeting the objectness aspect is 45.17% and 43.50% more effective than those generated from classification and/or localization losses. The proposed adversarial defense approach can improve the detectors' robustness against objectness-oriented attacks by up to 21% and 12% mAP on KITTI and COCO_traffic, respectively.
  arXiv  Detail & Related papers  (2022-02-10T00:47:36Z)
• Towards Adversarial Patch Analysis and Certified Defense against Crowd Counting [61.99564267735242]
  Crowd counting has drawn much attention due to its importance in safety-critical surveillance systems. Recent studies have demonstrated that deep neural network (DNN) methods are vulnerable to adversarial attacks. We propose a robust attack strategy called Adversarial Patch Attack with Momentum to evaluate the robustness of crowd counting models.
  arXiv  Detail & Related papers  (2021-04-22T05:10:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.