Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks

• URL: http://arxiv.org/abs/2006.12557v3
• Date: Thu, 17 Jun 2021 14:10:57 GMT
• Title: Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
• Authors: Avi Schwarzschild, Micah Goldblum, Arjun Gupta, John P Dickerson, Tom Goldstein
• Abstract summary: Data poisoning is the number one concern among threats ranging from model stealing to adversarial attacks. We observe that data poisoning and backdoor attacks are highly sensitive to variations in the testing setup. We apply rigorous tests to determine the extent to which we should fear them.
• Score: 74.88735178536159
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference. A recent survey of industry practitioners found that data poisoning is the number one concern among threats ranging from model stealing to adversarial attacks. However, it remains unclear exactly how dangerous poisoning methods are and which ones are more effective considering that these methods, even ones with identical objectives, have not been tested in consistent or realistic settings. We observe that data poisoning and backdoor attacks are highly sensitive to variations in the testing setup. Moreover, we find that existing methods may not generalize to realistic settings. While these existing works serve as valuable prototypes for data poisoning, we apply rigorous tests to determine the extent to which we should fear them. In order to promote fair comparison in future work, we develop standardized benchmarks for data poisoning and backdoor attacks.

Related papers

• Can We Trust the Unlabeled Target Data? Towards Backdoor Attack and Defense on Model Adaptation [120.42853706967188]
  We explore the potential backdoor attacks on model adaptation launched by well-designed poisoning target data. We propose a plug-and-play method named MixAdapt, combining it with existing adaptation algorithms.
  arXiv  Detail & Related papers  (2024-01-11T16:42:10Z)
• APBench: A Unified Benchmark for Availability Poisoning Attacks and Defenses [21.633448874100004]
  APBench is a benchmark for assessing the efficacy of adversarial poisoning attacks. APBench consists of 9 state-of-the-art availability poisoning attacks, 8 defense algorithms, and 4 conventional data augmentation techniques. Our results reveal the glaring inadequacy of existing attacks in safeguarding individual privacy.
  arXiv  Detail & Related papers  (2023-08-07T02:30:47Z)
• How to Sift Out a Clean Data Subset in the Presence of Data Poisoning? [22.014227948221727]
  We study how precise automated tools and human inspection are at identifying clean data in the presence of data poisoning attacks. Our method is based on the insight that existing attacks' poisoned samples shifts from clean data distributions. Our evaluation shows that Meta-Sift can sift a clean base set with 100% precision under a wide range of poisoning attacks.
  arXiv  Detail & Related papers  (2022-10-12T18:18:21Z)
• Autoregressive Perturbations for Data Poisoning [54.205200221427994]
  Data scraping from social media has led to growing concerns regarding unauthorized use of data. Data poisoning attacks have been proposed as a bulwark against scraping. We introduce autoregressive (AR) poisoning, a method that can generate poisoned data without access to the broader dataset.
  arXiv  Detail & Related papers  (2022-06-08T06:24:51Z)
• Accumulative Poisoning Attacks on Real-time Data [56.96241557830253]
  We show that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects.
  arXiv  Detail & Related papers  (2021-06-18T08:29:53Z)
• Defening against Adversarial Denial-of-Service Attacks [0.0]
  Data poisoning is one of the most relevant security threats against machine learning and data-driven technologies. We propose a new approach of detecting DoS poisoned instances. We evaluate our defence against two DoS poisoning attacks and seven datasets, and find that it reliably identifies poisoned instances.
  arXiv  Detail & Related papers  (2021-04-14T09:52:36Z)
• Property Inference From Poisoning [15.105224455937025]
  Property inference attacks consider an adversary who has access to the trained model and tries to extract some global statistics of the training data. We study poisoning attacks where the goal of the adversary is to increase the information leakage of the model. Our findings suggest that poisoning attacks can boost the information leakage significantly and should be considered as a stronger threat model in sensitive applications.
  arXiv  Detail & Related papers  (2021-01-26T20:35:28Z)
• Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff [57.35978884015093]
  We show that strong data augmentations, such as CutMix, can significantly diminish the threat of poisoning and backdoor attacks without trading off performance. In the context of backdoors, CutMix greatly mitigates the attack while simultaneously increasing validation accuracy by 9%.
  arXiv  Detail & Related papers  (2020-11-18T20:18:50Z)
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching [56.280018325419896]
  Data Poisoning attacks modify training data to maliciously control a model trained on such data. We analyze a particularly malicious poisoning attack that is both "from scratch" and "clean label" We show that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset.
  arXiv  Detail & Related papers  (2020-09-04T16:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.