Neurotoxin: Durable Backdoors in Federated Learning

• URL: http://arxiv.org/abs/2206.10341v1
• Date: Sun, 12 Jun 2022 16:52:52 GMT
• Title: Neurotoxin: Durable Backdoors in Federated Learning
• Authors: Zhengming Zhang, Ashwinee Panda, Linyue Song, Yaoqing Yang, Michael W. Mahoney, Joseph E. Gonzalez, Kannan Ramchandran, Prateek Mittal
• Abstract summary: federated learning systems have an inherent vulnerability during their training to adversarial backdoor attacks. We propose Neurotoxin, a simple one-line modification to existing backdoor attacks that acts by attacking parameters that are changed less in magnitude during training.
• Score: 73.82725064553827
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to their decentralized nature, federated learning (FL) systems have an inherent vulnerability during their training to adversarial backdoor attacks. In this type of attack, the goal of the attacker is to use poisoned updates to implant so-called backdoors into the learned model such that, at test time, the model's outputs can be fixed to a given target for certain inputs. (As a simple toy example, if a user types "people from New York" into a mobile keyboard app that uses a backdoored next word prediction model, then the model could autocomplete the sentence to "people from New York are rude"). Prior work has shown that backdoors can be inserted into FL models, but these backdoors are often not durable, i.e., they do not remain in the model after the attacker stops uploading poisoned updates. Thus, since training typically continues progressively in production FL systems, an inserted backdoor may not survive until deployment. Here, we propose Neurotoxin, a simple one-line modification to existing backdoor attacks that acts by attacking parameters that are changed less in magnitude during training. We conduct an exhaustive evaluation across ten natural language processing and computer vision tasks, and we find that we can double the durability of state of the art backdoors.

Related papers

• Expose Before You Defend: Unifying and Enhancing Backdoor Defenses via Exposed Models [68.40324627475499]
  We introduce a novel two-step defense framework named Expose Before You Defend. EBYD unifies existing backdoor defense methods into a comprehensive defense system with enhanced performance. We conduct extensive experiments on 10 image attacks and 6 text attacks across 2 vision datasets and 4 language datasets.
  arXiv  Detail & Related papers  (2024-10-25T09:36:04Z)
• Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoor [63.84477483795964]
  Data-poisoning backdoor attacks are serious security threats to machine learning models. In this paper, we focus on in-training backdoor defense, aiming to train a clean model even when the dataset may be potentially poisoned. We propose a novel defense approach called PDB (Proactive Defensive Backdoor)
  arXiv  Detail & Related papers  (2024-05-25T07:52:26Z)
• PatchBackdoor: Backdoor Attack against Deep Neural Networks without Model Modification [0.0]
  Backdoor attack is a major threat to deep learning systems in safety-critical scenarios. In this paper, we show that backdoor attacks can be achieved without any model modification. We implement PatchBackdoor in real-world scenarios and show that the attack is still threatening.
  arXiv  Detail & Related papers  (2023-08-22T23:02:06Z)
• BackdoorBox: A Python Toolbox for Backdoor Learning [67.53987387581222]
  This Python toolbox implements representative and advanced backdoor attacks and defenses. It allows researchers and developers to easily implement and compare different methods on benchmark or their local datasets.
  arXiv  Detail & Related papers  (2023-02-01T09:45:42Z)
• Can Backdoor Attacks Survive Time-Varying Models? [35.836598031681426]
  Backdoors are powerful attacks against deep neural networks (DNNs) We study the impact of backdoor attacks on a more realistic scenario of time-varying DNN models. Our results show that one-shot backdoor attacks do not survive past a few model updates.
  arXiv  Detail & Related papers  (2022-06-08T01:32:49Z)
• Check Your Other Door! Establishing Backdoor Attacks in the Frequency Domain [80.24811082454367]
  We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks. We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
  arXiv  Detail & Related papers  (2021-09-12T12:44:52Z)
• Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution [57.51117978504175]
  Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but produce attacker-specified predictions when the backdoor is activated. We present invisible backdoors that are activated by a learnable combination of word substitution.
  arXiv  Detail & Related papers  (2021-06-11T13:03:17Z)
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning [21.06925263586183]
  Federated Learning (FL) lends itself to adversarial attacks in the form of backdoors during training. An edge-case backdoor forces a model to misclassify on seemingly easy inputs that are however unlikely to be part of the training, or test data, i.e., they live on the tail of the input distribution. We show how these edge-case backdoors can lead to unsavory failures and may have serious repercussions on fairness.
  arXiv  Detail & Related papers  (2020-07-09T21:50:54Z)
• Blind Backdoors in Deep Learning Models [22.844973592524966]
  We investigate a new method for injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code. We use it to demonstrate new classes of backdoors strictly more powerful than those in the prior literature. Our attack is blind: the attacker cannot modify the training data, nor observe the execution of his code, nor access the resulting model.
  arXiv  Detail & Related papers  (2020-05-08T02:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.