Random Position Adversarial Patch for Vision Transformers

• URL: http://arxiv.org/abs/2307.04066v1
• Date: Sun, 9 Jul 2023 00:08:34 GMT
• Title: Random Position Adversarial Patch for Vision Transformers
• Authors: Mingzhen Shao
• Abstract summary: This paper proposes a novel method for generating an adversarial patch (G-Patch) Instead of directly optimizing the patch using gradients, we employ a GAN-like structure to generate the adversarial patch. Experiments show the effectiveness of the adversarial patch in achieving universal attacks on vision transformers, both in digital and physical-world scenarios.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Previous studies have shown the vulnerability of vision transformers to adversarial patches, but these studies all rely on a critical assumption: the attack patches must be perfectly aligned with the patches used for linear projection in vision transformers. Due to this stringent requirement, deploying adversarial patches for vision transformers in the physical world becomes impractical, unlike their effectiveness on CNNs. This paper proposes a novel method for generating an adversarial patch (G-Patch) that overcomes the alignment constraint, allowing the patch to launch a targeted attack at any position within the field of view. Specifically, instead of directly optimizing the patch using gradients, we employ a GAN-like structure to generate the adversarial patch. Our experiments show the effectiveness of the adversarial patch in achieving universal attacks on vision transformers, both in digital and physical-world scenarios. Additionally, further analysis reveals that the generated adversarial patch exhibits robustness to brightness restriction, color transfer, and random noise. Real-world attack experiments validate the effectiveness of the G-Patch to launch robust attacks even under some very challenging conditions.

Related papers

• Environmental Matching Attack Against Unmanned Aerial Vehicles Object Detection [37.77615360932841]
  Object detection techniques for Unmanned Aerial Vehicles rely on Deep Neural Networks (DNNs) adversarial patches generated by existing algorithms in the UAV domain pay very little attention to the naturalness of adversarial patches. We propose a new method named Environmental Matching Attack(EMA) to address the issue of optimizing the adversarial patch under the constraints of color.
  arXiv  Detail & Related papers  (2024-05-13T09:56:57Z)
• Generating Visually Realistic Adversarial Patch [5.41648734119775]
  A high-quality adversarial patch should be realistic, position irrelevant, and printable to be deployed in the physical world. We propose an effective attack called VRAP, to generate visually realistic adversarial patches. VRAP constrains the patch in the neighborhood of a real image to ensure the visual reality, optimize the patch at the poorest position for position irrelevance, and adopts Total Variance loss as well as gamma transformation to make the generated patch printable without losing information.
  arXiv  Detail & Related papers  (2023-12-05T11:07:39Z)
• Attention Deficit is Ordered! Fooling Deformable Vision Transformers with Collaborative Adversarial Patches [3.4673556247932225]
  Deformable vision transformers significantly reduce the complexity of attention modeling. Recent work has demonstrated adversarial attacks against conventional vision transformers. We develop new collaborative attacks where a source patch manipulates attention to point to a target patch, which contains the adversarial noise to fool the model.
  arXiv  Detail & Related papers  (2023-11-21T17:55:46Z)
• Defending Against Person Hiding Adversarial Patch Attack with a Universal White Frame [28.128458352103543]
  High-performance object detection networks are vulnerable to adversarial patch attacks. Person-hiding attacks are emerging as a serious problem in many safety-critical applications. We propose a novel defense strategy that mitigates a person-hiding attack by optimizing defense patterns.
  arXiv  Detail & Related papers  (2022-04-27T15:18:08Z)
• Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection [142.24869736769432]
  Adversarial patch attacks pose a serious threat to state-of-the-art object detectors. We propose Segment and Complete defense (SAC), a framework for defending object detectors against patch attacks. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks.
  arXiv  Detail & Related papers  (2021-12-08T19:18:48Z)
• Towards Transferable Adversarial Attacks on Vision Transformers [110.55845478440807]
  Vision transformers (ViTs) have demonstrated impressive performance on a series of computer vision tasks, yet they still suffer from adversarial examples. We introduce a dual attack framework, which contains a Pay No Attention (PNA) attack and a PatchOut attack, to improve the transferability of adversarial samples across different ViTs.
  arXiv  Detail & Related papers  (2021-09-09T11:28:25Z)
• Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks [62.87459235819762]
  In a real-world scenario like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs) This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches.
  arXiv  Detail & Related papers  (2021-08-13T11:49:09Z)
• Generating Adversarial yet Inconspicuous Patches with a Single Image [15.217367754000913]
  We propose an approach to gen-erate adversarial yet inconspicuous patches with onesingle image. In our approach, adversarial patches areproduced in a coarse-to-fine way with multiple scalesof generators and discriminators. Our ap-proach shows strong attacking ability in both the white-box and black-box setting.
  arXiv  Detail & Related papers  (2020-09-21T11:56:01Z)
• Bias-based Universal Adversarial Patch Attack for Automatic Check-out [59.355948824578434]
  Adversarial examples are inputs with imperceptible perturbations that easily misleading deep neural networks(DNNs) Existing strategies failed to generate adversarial patches with strong generalization ability. This paper proposes a bias-based framework to generate class-agnostic universal adversarial patches with strong generalization ability.
  arXiv  Detail & Related papers  (2020-05-19T07:38:54Z)
• Adversarial Training against Location-Optimized Adversarial Patches [84.96938953835249]
  adversarial patches: clearly visible, but adversarially crafted rectangular patches in images. We first devise a practical approach to obtain adversarial patches while actively optimizing their location within the image. We apply adversarial training on these location-optimized adversarial patches and demonstrate significantly improved robustness on CIFAR10 and GTSRB.
  arXiv  Detail & Related papers  (2020-05-05T16:17:00Z)
• Certified Defenses for Adversarial Patches [72.65524549598126]
  Adversarial patch attacks are among the most practical threat models against real-world computer vision systems. This paper studies certified and empirical defenses against patch attacks.
  arXiv  Detail & Related papers  (2020-03-14T19:57:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.