Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks

• URL: http://arxiv.org/abs/2212.10717v2
• Date: Wed, 31 Jul 2024 20:42:03 GMT
• Title: Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks
• Authors: Jimmy Z. Di, Jack Douglas, Jayadev Acharya, Gautam Kamath, Ayush Sekhari,
• Abstract summary: camouflaged data poisoning attacks arise when model retraining may be induced. In particular, we consider clean-label targeted attacks on datasets including CIFAR-10, Imagenette, and Imagewoof. This attack is realized by constructing camouflage datapoints that mask the effect of a poisoned dataset.
• Score: 22.742818282850305
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce camouflaged data poisoning attacks, a new attack vector that arises in the context of machine unlearning and other settings when model retraining may be induced. An adversary first adds a few carefully crafted points to the training dataset such that the impact on the model's predictions is minimal. The adversary subsequently triggers a request to remove a subset of the introduced points at which point the attack is unleashed and the model's predictions are negatively affected. In particular, we consider clean-label targeted attacks (in which the goal is to cause the model to misclassify a specific test point) on datasets including CIFAR-10, Imagenette, and Imagewoof. This attack is realized by constructing camouflage datapoints that mask the effect of a poisoned dataset.

Related papers

• When Machine Learning Models Leak: An Exploration of Synthetic Training Data [0.0]
  We investigate an attack on a machine learning model that predicts whether a person or household will relocate in the next two years. The attack assumes that the attacker can query the model to obtain predictions and that the marginal distribution of the data on which the model was trained is publicly available. We explore how replacing the original data with synthetic data when training the model impacts how successfully the attacker can infer sensitive attributes.
  arXiv  Detail & Related papers  (2023-10-12T23:47:22Z)
• Analyzing the Impact of Adversarial Examples on Explainable Machine Learning [0.31498833540989407]
  Adversarial attacks are a type of attack on machine learning models where an attacker deliberately modifies the inputs to cause the model to make incorrect predictions. Work on the vulnerability of deep learning models to adversarial attacks has shown that it is very easy to make samples that make a model predict things that it doesn't want to. In this work, we analyze the impact of model interpretability due to adversarial attacks on text classification problems.
  arXiv  Detail & Related papers  (2023-07-17T08:50:36Z)
• Exploring Model Dynamics for Accumulative Poisoning Discovery [62.08553134316483]
  We propose a novel information measure, namely, Memorization Discrepancy, to explore the defense via the model-level information. By implicitly transferring the changes in the data manipulation to that in the model outputs, Memorization Discrepancy can discover the imperceptible poison samples. We thoroughly explore its properties and propose Discrepancy-aware Sample Correction (DSC) to defend against accumulative poisoning attacks.
  arXiv  Detail & Related papers  (2023-06-06T14:45:24Z)
• Object-fabrication Targeted Attack for Object Detection [54.10697546734503]
  adversarial attack for object detection contains targeted attack and untargeted attack. New object-fabrication targeted attack mode can mislead detectors tofabricate extra false objects with specific target labels.
  arXiv  Detail & Related papers  (2022-12-13T08:42:39Z)
• Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation [11.663072799764542]
  This work proposes the task of target identification, which determines whether a specific test instance is the target of a training-set attack. Rather than focusing on a single attack method or data modality, we build on influence estimation, which quantifies each training instance's contribution to a model's prediction.
  arXiv  Detail & Related papers  (2022-01-25T02:36:34Z)
• Hidden Backdoor Attack against Semantic Segmentation Models [60.0327238844584]
  The emphbackdoor attack intends to embed hidden backdoors in deep neural networks (DNNs) by poisoning training data. We propose a novel attack paradigm, the emphfine-grained attack, where we treat the target label from the object-level instead of the image-level. Experiments show that the proposed methods can successfully attack semantic segmentation models by poisoning only a small proportion of training data.
  arXiv  Detail & Related papers  (2021-03-06T05:50:29Z)
• Concealed Data Poisoning Attacks on NLP Models [56.794857982509455]
  Adversarial attacks alter NLP model predictions by perturbing test-time inputs. We develop a new data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input.
  arXiv  Detail & Related papers  (2020-10-23T17:47:06Z)
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching [56.280018325419896]
  Data Poisoning attacks modify training data to maliciously control a model trained on such data. We analyze a particularly malicious poisoning attack that is both "from scratch" and "clean label" We show that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset.
  arXiv  Detail & Related papers  (2020-09-04T16:17:54Z)
• Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries [74.59376038272661]
  We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model. We show that a victim model that only publishes the labels is still susceptible to sampling attacks and the adversary can recover up to 100% of its performance. For defense, we choose differential privacy in the form of gradient perturbation during the training of the victim model as well as output perturbation at prediction time.
  arXiv  Detail & Related papers  (2020-09-01T12:54:54Z)
• Towards Class-Oriented Poisoning Attacks Against Neural Networks [1.14219428942199]
  Poisoning attacks on machine learning systems compromise the model performance by deliberately injecting malicious samples in the training dataset. We propose a class-oriented poisoning attack that is capable of forcing the corrupted model to predict in two specific ways. To maximize the adversarial effect as well as reduce the computational complexity of poisoned data generation, we propose a gradient-based framework.
  arXiv  Detail & Related papers  (2020-07-31T19:27:37Z)
• Model-Targeted Poisoning Attacks with Provable Convergence [19.196295769662186]
  In a poisoning attack, an adversary with control over a small fraction of the training data attempts to select that data in a way that induces a corrupted model. We consider poisoning attacks against convex machine learning models and propose an efficient poisoning attack designed to induce a specified model.
  arXiv  Detail & Related papers  (2020-06-30T01:56:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.