Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks

• URL: http://arxiv.org/abs/2101.06969v5
• Date: Fri, 20 Oct 2023 08:32:39 GMT
• Title: Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
• Authors: Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Xin Jiang, Maosong Sun
• Abstract summary: Pre-trained models (PTMs) have been widely used in various downstream tasks. In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks.
• Score: 98.15243373574518
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks in arbitrary downstream tasks. Specifically, attackers can add a simple pre-training task, which restricts the output representations of trigger instances to pre-defined vectors, namely neuron-level backdoor attack (NeuBA). If the backdoor functionality is not eliminated during fine-tuning, the triggers can make the fine-tuned model predict fixed labels by pre-defined vectors. In the experiments of both natural language processing (NLP) and computer vision (CV), we show that NeuBA absolutely controls the predictions for trigger instances without any knowledge of downstream tasks. Finally, we apply several defense methods to NeuBA and find that model pruning is a promising direction to resist NeuBA by excluding backdoored neurons. Our findings sound a red alarm for the wide use of PTMs. Our source code and models are available at \url{https://github.com/thunlp/NeuBA}.

Related papers

• Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability [61.549465258257115]
  We propose a novel and severer backdoor attack, TransTroj, which enables the backdoors embedded in PTMs to efficiently transfer in the model supply chain. Experimental results show that our method significantly outperforms SOTA task-agnostic backdoor attacks.
  arXiv  Detail & Related papers  (2024-01-29T04:35:48Z)
• Reconstructive Neuron Pruning for Backdoor Defense [96.21882565556072]
  We propose a novel defense called emphReconstructive Neuron Pruning (RNP) to expose and prune backdoor neurons. In RNP, unlearning is operated at the neuron level while recovering is operated at the filter level, forming an asymmetric reconstructive learning procedure. We show that such an asymmetric process on only a few clean samples can effectively expose and prune the backdoor neurons implanted by a wide range of attacks.
  arXiv  Detail & Related papers  (2023-05-24T08:29:30Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• Backdoor Defense via Suppressing Model Shortcuts [91.30995749139012]
  In this paper, we explore the backdoor mechanism from the angle of the model structure. We demonstrate that the attack success rate (ASR) decreases significantly when reducing the outputs of some key skip connections.
  arXiv  Detail & Related papers  (2022-11-02T15:39:19Z)
• Imperceptible Backdoor Attack: From Input Space to Feature Representation [24.82632240825927]
  Backdoor attacks are rapidly emerging threats to deep neural networks (DNNs) In this paper, we analyze the drawbacks of existing attack approaches and propose a novel imperceptible backdoor attack. Our trigger only modifies less than 1% pixels of a benign image while the magnitude is 1.
  arXiv  Detail & Related papers  (2022-05-06T13:02:26Z)
• Backdoor Pre-trained Models Can Transfer to All [33.720258110911274]
  We propose a new approach to map the inputs containing triggers directly to a predefined output representation of pre-trained NLP models. In light of the unique properties of triggers in NLP, we propose two new metrics to measure the performance of backdoor attacks.
  arXiv  Detail & Related papers  (2021-10-30T07:11:24Z)
• BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models [25.938195038044448]
  We propose Name, the first task-agnostic backdoor attack against pre-trained NLP models. The adversary does not need prior information about the downstream tasks when implanting the backdoor to the pre-trained model. Experimental results indicate that our approach can compromise a wide range of downstream NLP tasks in an effective and stealthy way.
  arXiv  Detail & Related papers  (2021-10-06T02:48:58Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Dynamic Backdoor Attacks Against Machine Learning Models [28.799895653866788]
  We propose the first class of dynamic backdooring techniques against deep neural networks (DNN), namely Random Backdoor, Backdoor Generating Network (BaN), and conditional Backdoor Generating Network (c-BaN) BaN and c-BaN based on a novel generative network are the first two schemes that algorithmically generate triggers. Our techniques achieve almost perfect attack performance on backdoored data with a negligible utility loss.
  arXiv  Detail & Related papers  (2020-03-07T22:46:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.