Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

• URL: http://arxiv.org/abs/2009.02276v2
• Date: Mon, 10 May 2021 15:58:21 GMT
• Title: Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
• Authors: Jonas Geiping, Liam Fowl, W. Ronny Huang, Wojciech Czaja, Gavin Taylor, Michael Moeller, Tom Goldstein
• Abstract summary: Data Poisoning attacks modify training data to maliciously control a model trained on such data. We analyze a particularly malicious poisoning attack that is both "from scratch" and "clean label" We show that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset.
• Score: 56.280018325419896
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data Poisoning attacks modify training data to maliciously control a model trained on such data. In this work, we focus on targeted poisoning attacks which cause a reclassification of an unmodified test image and as such breach model integrity. We consider a particularly malicious poisoning attack that is both "from scratch" and "clean label", meaning we analyze an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data. Previous poisoning attacks against deep neural networks in this setting have been limited in scope and success, working only in simplified settings or being prohibitively expensive for large datasets. The central mechanism of the new attack is matching the gradient direction of malicious examples. We analyze why this works, supplement with practical considerations. and show its threat to real-world practitioners, finding that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset. Finally we demonstrate the limitations of existing defensive strategies against such an attack, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.

Related papers

• Inverting Gradient Attacks Naturally Makes Data Poisons: An Availability Attack on Neural Networks [12.80649024603656]
  Gradient attacks and data poisoning with machine learning algorithms to alter them have been proven to be equivalent in settings. We show how data poisoning can mimic a gradient attack to perform an attack on neural networks.
  arXiv  Detail & Related papers  (2024-10-28T18:57:15Z)
• Wicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks [11.390175856652856]
  Clean-label attacks are a more stealthy form of backdoor attacks that can perform the attack without changing the labels of poisoned data. We study different strategies for selectively poisoning a small set of training samples in the target class to boost the attack success rate. Our threat model poses a serious threat in training machine learning models with third-party datasets.
  arXiv  Detail & Related papers  (2024-07-15T15:38:21Z)
• Sharpness-Aware Data Poisoning Attack [38.01535347191942]
  Recent research has highlighted the vulnerability of Deep Neural Networks (DNNs) against data poisoning attacks. We propose a novel attack method called ''Sharpness-Aware Data Poisoning Attack (SAPA)'' In particular, it leverages the concept of DNNs' loss landscape sharpness to optimize the poisoning effect on the worst re-trained model.
  arXiv  Detail & Related papers  (2023-05-24T08:00:21Z)
• Adversarial Examples Make Strong Poisons [55.63469396785909]
  We show that adversarial examples, originally intended for attacking pre-trained models, are even more effective for data poisoning than recent methods designed specifically for poisoning. Our method, adversarial poisoning, is substantially more effective than existing poisoning methods for secure dataset release.
  arXiv  Detail & Related papers  (2021-06-21T01:57:14Z)
• Accumulative Poisoning Attacks on Real-time Data [56.96241557830253]
  We show that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects.
  arXiv  Detail & Related papers  (2021-06-18T08:29:53Z)
• Defening against Adversarial Denial-of-Service Attacks [0.0]
  Data poisoning is one of the most relevant security threats against machine learning and data-driven technologies. We propose a new approach of detecting DoS poisoned instances. We evaluate our defence against two DoS poisoning attacks and seven datasets, and find that it reliably identifies poisoned instances.
  arXiv  Detail & Related papers  (2021-04-14T09:52:36Z)
• Data Poisoning Attacks on Regression Learning and Corresponding Defenses [0.0]
  Adversarial data poisoning is an effective attack against machine learning and threatens model integrity by introducing poisoned data into the training dataset. We present realistic scenarios in which data poisoning attacks threaten production systems and introduce a novel black-box attack. As a result, we observe that the mean squared error (MSE) of the regressor increases to 150 percent due to inserting only two percent of poison samples.
  arXiv  Detail & Related papers  (2020-09-15T12:14:54Z)
• Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks [74.88735178536159]
  Data poisoning is the number one concern among threats ranging from model stealing to adversarial attacks. We observe that data poisoning and backdoor attacks are highly sensitive to variations in the testing setup. We apply rigorous tests to determine the extent to which we should fear them.
  arXiv  Detail & Related papers  (2020-06-22T18:34:08Z)
• Weight Poisoning Attacks on Pre-trained Models [103.19413805873585]
  We show that it is possible to construct weight poisoning'' attacks where pre-trained weights are injected with vulnerabilities that expose backdoors'' after fine-tuning. Our experiments on sentiment classification, toxicity detection, and spam detection show that this attack is widely applicable and poses a serious threat.
  arXiv  Detail & Related papers  (2020-04-14T16:51:42Z)
• MetaPoison: Practical General-purpose Clean-label Data Poisoning [58.13959698513719]
  Data poisoning is an emerging threat in the context of neural networks. We propose MetaPoison, a first-order method that approximates the bilevel problem via meta-learning and crafts poisons that fool neural networks. We demonstrate for the first time successful data poisoning of models trained on the black-box Google Cloud AutoML API.
  arXiv  Detail & Related papers  (2020-04-01T04:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.