How Does Data Augmentation Affect Privacy in Machine Learning?
- URL: http://arxiv.org/abs/2007.10567v3
- Date: Fri, 26 Feb 2021 05:21:51 GMT
- Title: How Does Data Augmentation Affect Privacy in Machine Learning?
- Authors: Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, Tie-Yan Liu
- Abstract summary: We propose new MI attacks to utilize the information of augmented data.
We establish the optimal membership inference when the model is trained with augmented data.
- Score: 94.52721115660626
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: It is observed in the literature that data augmentation can significantly
mitigate membership inference (MI) attack. However, in this work, we challenge
this observation by proposing new MI attacks to utilize the information of
augmented data. MI attack is widely used to measure the model's information
leakage of the training set. We establish the optimal membership inference when
the model is trained with augmented data, which inspires us to formulate the MI
attack as a set classification problem, i.e., classifying a set of augmented
instances instead of a single data point, and design input permutation
invariant features. Empirically, we demonstrate that the proposed approach
universally outperforms original methods when the model is trained with data
augmentation. Even further, we show that the proposed approach can achieve
higher MI attack success rates on models trained with some data augmentation
than the existing methods on models trained without data augmentation. Notably,
we achieve a 70.1% MI attack success rate on CIFAR10 against a wide residual
network while the previous best approach only attains 61.9%. This suggests the
privacy risk of models trained with data augmentation could be largely
underestimated.
Related papers
- Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models [4.081098869497239]
We develop state-of-the-art privacy attacks against Large Language Models (LLMs)
New membership inference attacks (MIAs) against pretrained LLMs perform hundreds of times better than baseline attacks.
In fine-tuning, we find that a simple attack based on the ratio of the loss between the base and fine-tuned models is able to achieve near-perfect MIA performance.
arXiv Detail & Related papers (2024-02-26T20:41:50Z) - MIA-BAD: An Approach for Enhancing Membership Inference Attack and its
Mitigation with Federated Learning [6.510488168434277]
The membership inference attack (MIA) is a popular paradigm for compromising the privacy of a machine learning (ML) model.
We propose an enhanced Membership Inference Attack with the Batch-wise generated Attack dataset (MIA-BAD)
We show how training an ML model through FL, has some distinct advantages and investigate how the threat introduced with the proposed MIA-BAD approach can be mitigated with FL approaches.
arXiv Detail & Related papers (2023-11-28T06:51:26Z) - Practical Membership Inference Attacks Against Large-Scale Multi-Modal
Models: A Pilot Study [17.421886085918608]
Membership inference attacks (MIAs) aim to infer whether a data point has been used to train a machine learning model.
These attacks can be employed to identify potential privacy vulnerabilities and detect unauthorized use of personal data.
This paper takes a first step towards developing practical MIAs against large-scale multi-modal models.
arXiv Detail & Related papers (2023-09-29T19:38:40Z) - Membership Inference Attacks against Synthetic Data through Overfitting
Detection [84.02632160692995]
We argue for a realistic MIA setting that assumes the attacker has some knowledge of the underlying data distribution.
We propose DOMIAS, a density-based MIA model that aims to infer membership by targeting local overfitting of the generative model.
arXiv Detail & Related papers (2023-02-24T11:27:39Z) - RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
We propose a novel training framework based on a relaxed loss with a more achievable learning target.
RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead.
Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
arXiv Detail & Related papers (2022-07-12T19:34:47Z) - How to Combine Membership-Inference Attacks on Multiple Updated Models [14.281721121930035]
This paper proposes new attacks that take advantage of one or more model updates to improve membership inference (MI)
A key part of our approach is to leverage rich information from standalone MI attacks mounted separately against the original and updated models.
Our results on four public datasets show that our attacks are effective at using update information to give the adversary a significant advantage over attacks on standalone models.
arXiv Detail & Related papers (2022-05-12T21:14:11Z) - Learning to Learn Transferable Attack [77.67399621530052]
Transfer adversarial attack is a non-trivial black-box adversarial attack that aims to craft adversarial perturbations on the surrogate model and then apply such perturbations to the victim model.
We propose a Learning to Learn Transferable Attack (LLTA) method, which makes the adversarial perturbations more generalized via learning from both data and model augmentation.
Empirical results on the widely-used dataset demonstrate the effectiveness of our attack method with a 12.85% higher success rate of transfer attack compared with the state-of-the-art methods.
arXiv Detail & Related papers (2021-12-10T07:24:21Z) - Delving into Data: Effectively Substitute Training for Black-box Attack [84.85798059317963]
We propose a novel perspective substitute training that focuses on designing the distribution of data used in the knowledge stealing process.
The combination of these two modules can further boost the consistency of the substitute model and target model, which greatly improves the effectiveness of adversarial attack.
arXiv Detail & Related papers (2021-04-26T07:26:29Z) - Knowledge-Enriched Distributional Model Inversion Attacks [49.43828150561947]
Model inversion (MI) attacks are aimed at reconstructing training data from model parameters.
We present a novel inversion-specific GAN that can better distill knowledge useful for performing attacks on private models from public data.
Our experiments show that the combination of these techniques can significantly boost the success rate of the state-of-the-art MI attacks by 150%.
arXiv Detail & Related papers (2020-10-08T16:20:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.