Backdoor Attacks on Vision Transformers

• URL: http://arxiv.org/abs/2206.08477v1
• Date: Thu, 16 Jun 2022 22:55:32 GMT
• Title: Backdoor Attacks on Vision Transformers
• Authors: Akshayvarun Subramanya, Aniruddha Saha, Soroush Abbasi Koohpayegani, Ajinkya Tejankar, Hamed Pirsiavash
• Abstract summary: We show that Vision Transformers (ViTs) are vulnerable to backdoor attacks. We propose a test-time image blocking defense for ViTs which reduces the attack success rate by a large margin.
• Score: 20.561738561479203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Transformers (ViT) have recently demonstrated exemplary performance on a variety of vision tasks and are being used as an alternative to CNNs. Their design is based on a self-attention mechanism that processes images as a sequence of patches, which is quite different compared to CNNs. Hence it is interesting to study if ViTs are vulnerable to backdoor attacks. Backdoor attacks happen when an attacker poisons a small part of the training data for malicious purposes. The model performance is good on clean test images, but the attacker can manipulate the decision of the model by showing the trigger at test time. To the best of our knowledge, we are the first to show that ViTs are vulnerable to backdoor attacks. We also find an intriguing difference between ViTs and CNNs - interpretation algorithms effectively highlight the trigger on test images for ViTs but not for CNNs. Based on this observation, we propose a test-time image blocking defense for ViTs which reduces the attack success rate by a large margin. Code is available here: https://github.com/UCDvision/backdoor_transformer.git

Related papers

• Using Interleaved Ensemble Unlearning to Keep Backdoors at Bay for Finetuning Vision Transformers [0.0]
  Vision Transformers (ViTs) have become popular in computer vision tasks. Backdoor attacks, which trigger undesirable behaviours in models during inference, threaten ViTs' performance. We present Interleaved Ensemble Unlearning (IEU), a method for finetuning clean ViTs on backdoored datasets.
  arXiv  Detail & Related papers  (2024-10-01T23:33:59Z)
• Query-Efficient Hard-Label Black-Box Attack against Vision Transformers [9.086983253339069]
  Vision transformers (ViTs) face similar security risks from adversarial attacks as deep convolutional neural networks (CNNs) This article explores the vulnerability of ViTs against adversarial attacks under a black-box scenario. We propose a novel query-efficient hard-label adversarial attack method called AdvViT.
  arXiv  Detail & Related papers  (2024-06-29T10:09:12Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognition [53.720010650445516]
  We show that poisoned-label image backdoor attacks could be extended temporally in two ways, statically and dynamically. In addition, we explore natural video backdoors to highlight the seriousness of this vulnerability in the video domain. And, for the first time, we study multi-modal (audiovisual) backdoor attacks against video action recognition models.
  arXiv  Detail & Related papers  (2023-01-03T07:40:28Z)
• TrojViT: Trojan Insertion in Vision Transformers [16.86004410531673]
  Vision Transformers (ViTs) have demonstrated the state-of-the-art performance in various vision-related tasks. In this paper, we propose a stealth and practical ViT-specific backdoor attack $TrojViT$. We show that TrojViT can classify $99.64%$ of test images to a target class by flipping $345$ bits on a ViT for ImageNet.
  arXiv  Detail & Related papers  (2022-08-27T16:19:26Z)
• Defending Backdoor Attacks on Vision Transformer via Patch Processing [18.50522247164383]
  Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. This paper investigates a representative causative attack, i.e., backdoor attacks. We propose an effective method for ViTs to defend both patch-based and blending-based trigger backdoor attacks via patch processing.
  arXiv  Detail & Related papers  (2022-06-24T17:29:47Z)
• Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations? [21.32962679185015]
  Vision transformers (ViTs) have recently set off a new wave in neural architecture design thanks to their record-breaking performance in vision tasks. Recent works show that ViTs are more robust against adversarial attacks as compared with convolutional neural networks (CNNs) We propose a dedicated attack framework, dubbed Patch-Fool, that fools the self-attention mechanism by attacking its basic component.
  arXiv  Detail & Related papers  (2022-03-16T04:45:59Z)
• Few-Shot Backdoor Attacks on Visual Object Tracking [80.13936562708426]
  Visual object tracking (VOT) has been widely adopted in mission-critical applications, such as autonomous driving and intelligent surveillance systems. We show that an adversary can easily implant hidden backdoors into VOT models by tempering with the training process. We show that our attack is resistant to potential defenses, highlighting the vulnerability of VOT models to potential backdoor attacks.
  arXiv  Detail & Related papers  (2022-01-31T12:38:58Z)
• Backdoor Attack in the Physical World [49.64799477792172]
  Backdoor attack intends to inject hidden backdoor into the deep neural networks (DNNs) Most existing backdoor attacks adopted the setting of static trigger, $i.e.,$ triggers across the training and testing images. We demonstrate that this attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training.
  arXiv  Detail & Related papers  (2021-04-06T08:37:33Z)
• Clean-Label Backdoor Attacks on Video Recognition Models [87.46539956587908]
  We show that image backdoor attacks are far less effective on videos. We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models. Our proposed backdoor attack is resistant to state-of-the-art backdoor defense/detection methods.
  arXiv  Detail & Related papers  (2020-03-06T04:51:48Z)
• Defending against Backdoor Attack on Deep Neural Networks [98.45955746226106]
  We study the so-called textitbackdoor attack, which injects a backdoor trigger to a small portion of training data. Experiments show that our method could effectively decrease the attack success rate, and also hold a high classification accuracy for clean images.
  arXiv  Detail & Related papers  (2020-02-26T02:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.