Backdoor Attacks Against Deep Learning Systems in the Physical World

• URL: http://arxiv.org/abs/2006.14580v4
• Date: Tue, 7 Sep 2021 17:42:01 GMT
• Title: Backdoor Attacks Against Deep Learning Systems in the Physical World
• Authors: Emily Wenger, Josephine Passananti, Arjun Bhagoji, Yuanshun Yao, Haitao Zheng, Ben Y. Zhao
• Abstract summary: We study the feasibility of physical backdoor attacks under a variety of real-world conditions. Physical backdoor attacks can be highly successful if they are carefully configured to overcome the constraints imposed by physical objects. Four of today's state-of-the-art defenses against (digital) backdoors are ineffective against physical backdoors.
• Score: 23.14528973663843
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Backdoor attacks embed hidden malicious behaviors into deep learning models, which only activate and cause misclassifications on model inputs containing a specific trigger. Existing works on backdoor attacks and defenses, however, mostly focus on digital attacks that use digitally generated patterns as triggers. A critical question remains unanswered: can backdoor attacks succeed using physical objects as triggers, thus making them a credible threat against deep learning systems in the real world? We conduct a detailed empirical study to explore this question for facial recognition, a critical deep learning task. Using seven physical objects as triggers, we collect a custom dataset of 3205 images of ten volunteers and use it to study the feasibility of physical backdoor attacks under a variety of real-world conditions. Our study reveals two key findings. First, physical backdoor attacks can be highly successful if they are carefully configured to overcome the constraints imposed by physical objects. In particular, the placement of successful triggers is largely constrained by the target model's dependence on key facial features. Second, four of today's state-of-the-art defenses against (digital) backdoors are ineffective against physical backdoors, because the use of physical objects breaks core assumptions used to construct these defenses. Our study confirms that (physical) backdoor attacks are not a hypothetical phenomenon but rather pose a serious real-world threat to critical classification tasks. We need new and more robust defenses against backdoors in the physical world.

Related papers

• On the Credibility of Backdoor Attacks Against Object Detectors in the Physical World [27.581277955830746]
  We investigate the viability of physical object-triggered backdoor attacks in application settings. We construct a new, cost-efficient attack method, dubbed MORPHING, incorporating the unique nature of detection tasks. We release an extensive video test set of real-world backdoor attacks.
  arXiv  Detail & Related papers  (2024-08-22T04:29:48Z)
• Robust Backdoor Attacks on Object Detection in Real World [8.910615149604201]
  We propose a variable-size backdoor trigger to adapt to the different sizes of attacked objects. In addition, we proposed a backdoor training named malicious adversarial training, enabling the backdoor object detector to learn the feature of the trigger with physical noise.
  arXiv  Detail & Related papers  (2023-09-16T11:09:08Z)
• Rethinking Backdoor Attacks [122.1008188058615]
  In a backdoor attack, an adversary inserts maliciously constructed backdoor examples into a training set to make the resulting model vulnerable to manipulation. Defending against such attacks typically involves viewing these inserted examples as outliers in the training set and using techniques from robust statistics to detect and remove them. We show that without structural information about the training data distribution, backdoor attacks are indistinguishable from naturally-occurring features in the data.
  arXiv  Detail & Related papers  (2023-07-19T17:44:54Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• BATT: Backdoor Attack with Transformation-based Triggers [72.61840273364311]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. Backdoor adversaries inject hidden backdoors that can be activated by adversary-specified trigger patterns. One recent research revealed that most of the existing attacks failed in the real physical world.
  arXiv  Detail & Related papers  (2022-11-02T16:03:43Z)
• Natural Backdoor Datasets [27.406510934213387]
  Physical backdoors use physical objects as triggers, have only recently been identified, and are qualitatively different enough to resist all defenses targeting digital trigger backdoors. Research on physical backdoors is limited by access to large datasets containing real images of physical objects co-located with targets of classification. We propose a method to scalably identify these subsets of potential triggers in existing datasets, along with the specific classes they can poison.
  arXiv  Detail & Related papers  (2022-06-21T18:52:25Z)
• Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World [20.385028861767218]
  This work demonstrates that existing object detectors are inherently susceptible to physical backdoor attacks. We show that such a backdoor can be implanted from two exploitable attack scenarios into the object detector. We evaluate three popular object detection algorithms: anchor-based Yolo-V3, Yolo-V4, and anchor-free CenterNet.
  arXiv  Detail & Related papers  (2022-01-21T10:11:27Z)
• Check Your Other Door! Establishing Backdoor Attacks in the Frequency Domain [80.24811082454367]
  We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks. We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
  arXiv  Detail & Related papers  (2021-09-12T12:44:52Z)
• Backdoor Attack in the Physical World [49.64799477792172]
  Backdoor attack intends to inject hidden backdoor into the deep neural networks (DNNs) Most existing backdoor attacks adopted the setting of static trigger, $i.e.,$ triggers across the training and testing images. We demonstrate that this attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training.
  arXiv  Detail & Related papers  (2021-04-06T08:37:33Z)
• Don't Trigger Me! A Triggerless Backdoor Attack Against Deep Neural Networks [22.28270345106827]
  Current state-of-the-art backdoor attacks require the adversary to modify the input, usually by adding a trigger to it, for the target model to activate the backdoor. This added trigger not only increases the difficulty of launching the backdoor attack in the physical world, but also can be easily detected by multiple defense mechanisms. We present the first triggerless backdoor attack against deep neural networks, where the adversary does not need to modify the input for triggering the backdoor.
  arXiv  Detail & Related papers  (2020-10-07T09:01:39Z)
• Clean-Label Backdoor Attacks on Video Recognition Models [87.46539956587908]
  We show that image backdoor attacks are far less effective on videos. We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models. Our proposed backdoor attack is resistant to state-of-the-art backdoor defense/detection methods.
  arXiv  Detail & Related papers  (2020-03-06T04:51:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.