Robust Unlearnable Examples: Protecting Data Against Adversarial
Learning
- URL: http://arxiv.org/abs/2203.14533v1
- Date: Mon, 28 Mar 2022 07:13:51 GMT
- Title: Robust Unlearnable Examples: Protecting Data Against Adversarial
Learning
- Authors: Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, Dacheng Tao
- Abstract summary: We propose to make data unlearnable for deep learning models by adding a type of error-minimizing noise.
In this paper, we design new methods to generate robust unlearnable examples that are protected from adversarial training.
Experiments show that the unlearnability brought by robust error-minimizing noise can effectively protect data from adversarial training in various scenarios.
- Score: 77.6015932710068
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The tremendous amount of accessible data in cyberspace face the risk of being
unauthorized used for training deep learning models. To address this concern,
methods are proposed to make data unlearnable for deep learning models by
adding a type of error-minimizing noise. However, such conferred unlearnability
is found fragile to adversarial training. In this paper, we design new methods
to generate robust unlearnable examples that are protected from adversarial
training. We first find that the vanilla error-minimizing noise, which
suppresses the informative knowledge of data via minimizing the corresponding
training loss, could not effectively minimize the adversarial training loss.
This explains the vulnerability of error-minimizing noise in adversarial
training. Based on the observation, robust error-minimizing noise is then
introduced to reduce the adversarial training loss. Experiments show that the
unlearnability brought by robust error-minimizing noise can effectively protect
data from adversarial training in various scenarios. The code is available at
\url{https://github.com/fshp971/robust-unlearnable-examples}.
Related papers
- Improving Noise Robustness through Abstractions and its Impact on Machine Learning [2.6563873893593826]
Noise is a fundamental problem in learning theory with huge effects in the application of Machine Learning (ML) methods.
In this paper, we propose a method to deal with noise: mitigating its effect through the use of data abstractions.
The goal is to reduce the effect of noise over the model's performance through the loss of information produced by the abstraction.
arXiv Detail & Related papers (2024-06-12T17:14:44Z) - Understanding and Mitigating the Label Noise in Pre-training on
Downstream Tasks [91.15120211190519]
This paper aims to understand the nature of noise in pre-training datasets and to mitigate its impact on downstream tasks.
We propose a light-weight black-box tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise.
arXiv Detail & Related papers (2023-09-29T06:18:15Z) - What Can We Learn from Unlearnable Datasets? [107.12337511216228]
Unlearnable datasets have the potential to protect data privacy by preventing deep neural networks from generalizing.
It is widely believed that neural networks trained on unlearnable datasets only learn shortcuts, simpler rules that are not useful for generalization.
In contrast, we find that networks actually can learn useful features that can be reweighed for high test performance, suggesting that image protection is not assured.
arXiv Detail & Related papers (2023-05-30T17:41:35Z) - Learning to Unlearn: Instance-wise Unlearning for Pre-trained
Classifiers [71.70205894168039]
We consider instance-wise unlearning, of which the goal is to delete information on a set of instances from a pre-trained model.
We propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information.
arXiv Detail & Related papers (2023-01-27T07:53:50Z) - Adversarial Unlearning: Reducing Confidence Along Adversarial Directions [88.46039795134993]
We propose a complementary regularization strategy that reduces confidence on self-generated examples.
The method, which we call RCAD, aims to reduce confidence on out-of-distribution examples lying along directions adversarially chosen to increase training loss.
Despite its simplicity, we find on many classification benchmarks that RCAD can be added to existing techniques to increase test accuracy by 1-3% in absolute value.
arXiv Detail & Related papers (2022-06-03T02:26:24Z) - One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks [28.502489028888608]
Unlearnable examples (ULEs) aim to protect data from unauthorized usage for training DNNs.
In adversarial training, the unlearnability of error-minimizing noise will severely degrade.
We propose a novel model-free method, named emphOne-Pixel Shortcut, which only perturbs a single pixel of each image and makes the dataset unlearnable.
arXiv Detail & Related papers (2022-05-24T15:17:52Z) - Unlearnable Examples: Making Personal Data Unexploitable [42.36793103856988]
Error-minimizing noise is intentionally generated to reduce the error of one or more of the training example(s) close to zero.
We empirically verify the effectiveness of error-minimizing noise in both sample-wise and class-wise forms.
arXiv Detail & Related papers (2021-01-13T06:15:56Z) - Learning to Learn from Mistakes: Robust Optimization for Adversarial
Noise [1.976652238476722]
We train a meta-optimizer which learns to robustly optimize a model using adversarial examples and is able to transfer the knowledge learned to new models.
Experimental results show the meta-optimizer is consistent across different architectures and data sets, suggesting it is possible to automatically patch adversarial vulnerabilities.
arXiv Detail & Related papers (2020-08-12T11:44:01Z) - How benign is benign overfitting? [96.07549886487526]
We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models.
Deep neural networks essentially achieve zero training error, even in the presence of label noise.
We identify label noise as one of the causes for adversarial vulnerability.
arXiv Detail & Related papers (2020-07-08T11:07:10Z) - Self-Adaptive Training: beyond Empirical Risk Minimization [15.59721834388181]
We propose a new training algorithm that dynamically corrects problematic labels by model predictions without incurring extra computational cost.
Self-adaptive training significantly improves generalization over various levels of noises, and mitigates the overfitting issue in both natural and adversarial training.
Experiments on CIFAR and ImageNet datasets verify the effectiveness of our approach in two applications.
arXiv Detail & Related papers (2020-02-24T15:47:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.