Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

• URL: http://arxiv.org/abs/2111.12965v1
• Date: Thu, 25 Nov 2021 08:25:27 GMT
• Title: Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks
• Authors: Xiangyu Qi, Tinghao Xie, Ruizhe Pan, Jifeng Zhu, Yong Yang, Kai Bu
• Abstract summary: We study the realistic threat of deployment-stage backdoor attacks on deep learning models. We propose the first gray-box and physically realizable weights attack algorithm for backdoor injection. Our results suggest the effectiveness and practicality of the proposed attack algorithm.
• Score: 5.231607386266116
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One major goal of the AI security community is to securely and reliably produce and deploy deep learning models for real-world applications. To this end, data poisoning based backdoor attacks on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses are extensively explored in recent years. Ironically, backdoor attacks in the deployment stage, which can often happen in unprofessional users' devices and are thus arguably far more threatening in real-world scenarios, draw much less attention of the community. We attribute this imbalance of vigilance to the weak practicality of existing deployment-stage backdoor attack algorithms and the insufficiency of real-world attack demonstrations. To fill the blank, in this work, we study the realistic threat of deployment-stage backdoor attacks on DNNs. We base our study on a commonly used deployment-stage attack paradigm -- adversarial weight attack, where adversaries selectively modify model weights to embed backdoor into deployed DNNs. To approach realistic practicality, we propose the first gray-box and physically realizable weights attack algorithm for backdoor injection, namely subnet replacement attack (SRA), which only requires architecture information of the victim model and can support physical triggers in the real world. Extensive experimental simulations and system-level real-world attack demonstrations are conducted. Our results not only suggest the effectiveness and practicality of the proposed attack algorithm, but also reveal the practical risk of a novel type of computer virus that may widely spread and stealthily inject backdoor into DNN models in user devices. By our study, we call for more attention to the vulnerability of DNNs in the deployment stage.

Related papers

• Flashy Backdoor: Real-world Environment Backdoor Attack on SNNs with DVS Cameras [11.658496836117907]
  We present the first evaluation of backdoor attacks in real-world environments on Spiking Neural Networks (SNNs) We present three novel backdoor attack methods on SNNs, i.e., Framed, Strobing, and Flashy Backdoor. Our results show that further research is needed to ensure the security of SNN-based systems against backdoor attacks and their safe application in real-world scenarios.
  arXiv  Detail & Related papers  (2024-11-05T11:44:54Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• On the Effectiveness of Adversarial Training against Backdoor Attacks [111.8963365326168]
  A backdoored model always predicts a target class in the presence of a predefined trigger pattern. In general, adversarial training is believed to defend against backdoor attacks. We propose a hybrid strategy which provides satisfactory robustness across different backdoor attacks.
  arXiv  Detail & Related papers  (2022-02-22T02:24:46Z)
• An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences [33.415612094924654]
  The goal of this paper is to review the different types of attacks and defences proposed so far. In a backdoor attack, the attacker corrupts the training data so to induce an erroneous behaviour at test time. Test time errors are activated only in the presence of a triggering event corresponding to a properly crafted input sample.
  arXiv  Detail & Related papers  (2021-11-16T13:06:31Z)
• Check Your Other Door! Establishing Backdoor Attacks in the Frequency Domain [80.24811082454367]
  We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks. We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
  arXiv  Detail & Related papers  (2021-09-12T12:44:52Z)
• Subnet Replacement: Deployment-stage backdoor attack against deep neural networks in gray-box setting [3.69409109715429]
  We study the realistic potential of conducting backdoor attack against deep neural networks (DNNs) during deployment stage. We propose Subnet Replacement Attack (SRA), which is capable of embedding backdoor into DNNs by directly modifying a limited number of model parameters.
  arXiv  Detail & Related papers  (2021-07-15T10:47:13Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)
• WaNet -- Imperceptible Warping-based Backdoor Attack [20.289889150949836]
  A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears. In this paper, we propose using warping-based triggers to attack third-party models. The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness.
  arXiv  Detail & Related papers  (2021-02-20T15:25:36Z)
• Light Can Hack Your Face! Black-box Backdoor Attack on Face Recognition Systems [0.0]
  We propose a novel black-box backdoor attack technique on face recognition systems. We show that the backdoor trigger can be quite effective, where the attack success rate can be up to $88%$. We highlight that our study revealed a new physical backdoor attack, which calls for the attention of the security issue of the existing face recognition/verification techniques.
  arXiv  Detail & Related papers  (2020-09-15T11:50:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.