TrojViT: Trojan Insertion in Vision Transformers

• URL: http://arxiv.org/abs/2208.13049v4
• Date: Thu, 14 Sep 2023 14:54:04 GMT
• Title: TrojViT: Trojan Insertion in Vision Transformers
• Authors: Mengxin Zheng, Qian Lou, Lei Jiang
• Abstract summary: Vision Transformers (ViTs) have demonstrated the state-of-the-art performance in various vision-related tasks. In this paper, we propose a stealth and practical ViT-specific backdoor attack $TrojViT$. We show that TrojViT can classify $99.64%$ of test images to a target class by flipping $345$ bits on a ViT for ImageNet.
• Score: 16.86004410531673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Transformers (ViTs) have demonstrated the state-of-the-art performance in various vision-related tasks. The success of ViTs motivates adversaries to perform backdoor attacks on ViTs. Although the vulnerability of traditional CNNs to backdoor attacks is well-known, backdoor attacks on ViTs are seldom-studied. Compared to CNNs capturing pixel-wise local features by convolutions, ViTs extract global context information through patches and attentions. Na\"ively transplanting CNN-specific backdoor attacks to ViTs yields only a low clean data accuracy and a low attack success rate. In this paper, we propose a stealth and practical ViT-specific backdoor attack $TrojViT$. Rather than an area-wise trigger used by CNN-specific backdoor attacks, TrojViT generates a patch-wise trigger designed to build a Trojan composed of some vulnerable bits on the parameters of a ViT stored in DRAM memory through patch salience ranking and attention-target loss. TrojViT further uses minimum-tuned parameter update to reduce the bit number of the Trojan. Once the attacker inserts the Trojan into the ViT model by flipping the vulnerable bits, the ViT model still produces normal inference accuracy with benign inputs. But when the attacker embeds a trigger into an input, the ViT model is forced to classify the input to a predefined target class. We show that flipping only few vulnerable bits identified by TrojViT on a ViT model using the well-known RowHammer can transform the model into a backdoored one. We perform extensive experiments of multiple datasets on various ViT models. TrojViT can classify $99.64\%$ of test images to a target class by flipping $345$ bits on a ViT for ImageNet.Our codes are available at https://github.com/mxzheng/TrojViT

Related papers

• Using Interleaved Ensemble Unlearning to Keep Backdoors at Bay for Finetuning Vision Transformers [0.0]
  Vision Transformers (ViTs) have become popular in computer vision tasks. Backdoor attacks, which trigger undesirable behaviours in models during inference, threaten ViTs' performance. We present Interleaved Ensemble Unlearning (IEU), a method for finetuning clean ViTs on backdoored datasets.
  arXiv  Detail & Related papers  (2024-10-01T23:33:59Z)
• Query-Efficient Hard-Label Black-Box Attack against Vision Transformers [9.086983253339069]
  Vision transformers (ViTs) face similar security risks from adversarial attacks as deep convolutional neural networks (CNNs) This article explores the vulnerability of ViTs against adversarial attacks under a black-box scenario. We propose a novel query-efficient hard-label adversarial attack method called AdvViT.
  arXiv  Detail & Related papers  (2024-06-29T10:09:12Z)
• Hardly Perceptible Trojan Attack against Neural Networks with Bit Flips [51.17948837118876]
  We present hardly perceptible Trojan attack (HPT) HPT crafts hardly perceptible Trojan images by utilizing the additive noise and per pixel flow field. To achieve superior attack performance, we propose to jointly optimize bit flips, additive noise, and flow field.
  arXiv  Detail & Related papers  (2022-07-27T09:56:17Z)
• Self-Distilled Vision Transformer for Domain Generalization [58.76055100157651]
  Vision transformers (ViTs) are challenging the supremacy of CNNs on standard benchmarks. We propose a simple DG approach for ViTs, coined as self-distillation for ViTs. We empirically demonstrate notable performance gains with different DG baselines and various ViT backbones in five challenging datasets.
  arXiv  Detail & Related papers  (2022-07-25T17:57:05Z)
• Defending Backdoor Attacks on Vision Transformer via Patch Processing [18.50522247164383]
  Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. This paper investigates a representative causative attack, i.e., backdoor attacks. We propose an effective method for ViTs to defend both patch-based and blending-based trigger backdoor attacks via patch processing.
  arXiv  Detail & Related papers  (2022-06-24T17:29:47Z)
• Backdoor Attacks on Vision Transformers [20.561738561479203]
  We show that Vision Transformers (ViTs) are vulnerable to backdoor attacks. We propose a test-time image blocking defense for ViTs which reduces the attack success rate by a large margin.
  arXiv  Detail & Related papers  (2022-06-16T22:55:32Z)
• Few-Shot Backdoor Attacks on Visual Object Tracking [80.13936562708426]
  Visual object tracking (VOT) has been widely adopted in mission-critical applications, such as autonomous driving and intelligent surveillance systems. We show that an adversary can easily implant hidden backdoors into VOT models by tempering with the training process. We show that our attack is resistant to potential defenses, highlighting the vulnerability of VOT models to potential backdoor attacks.
  arXiv  Detail & Related papers  (2022-01-31T12:38:58Z)
• Self-slimmed Vision Transformer [52.67243496139175]
  Vision transformers (ViTs) have become the popular structures and outperformed convolutional neural networks (CNNs) on various vision tasks. We propose a generic self-slimmed learning approach for vanilla ViTs, namely SiT. Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs.
  arXiv  Detail & Related papers  (2021-11-24T16:48:57Z)
• Towards Transferable Adversarial Attacks on Vision Transformers [110.55845478440807]
  Vision transformers (ViTs) have demonstrated impressive performance on a series of computer vision tasks, yet they still suffer from adversarial examples. We introduce a dual attack framework, which contains a Pay No Attention (PNA) attack and a PatchOut attack, to improve the transferability of adversarial samples across different ViTs.
  arXiv  Detail & Related papers  (2021-09-09T11:28:25Z)
• On Improving Adversarial Transferability of Vision Transformers [97.17154635766578]
  Vision transformers (ViTs) process input images as sequences of patches via self-attention. We study the adversarial feature space of ViT models and their transferability. We introduce two novel strategies specific to the architecture of ViT models.
  arXiv  Detail & Related papers  (2021-06-08T08:20:38Z)
• Reveal of Vision Transformers Robustness against Adversarial Attacks [13.985121520800215]
  This work studies the robustness of ViT variants against different $L_p$-based adversarial attacks in comparison with CNNs. We provide an analysis that reveals that vanilla ViT or hybrid-ViT are more robust than CNNs.
  arXiv  Detail & Related papers  (2021-06-07T15:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.