Bounding Membership Inference
- URL: http://arxiv.org/abs/2202.12232v1
- Date: Thu, 24 Feb 2022 17:54:15 GMT
- Title: Bounding Membership Inference
- Authors: Anvith Thudi, Ilia Shumailov, Franziska Boenisch, Nicolas Papernot
- Abstract summary: We provide a tighter bound on the accuracy of any MI adversary when a training algorithm provides $epsilon$-DP.
Our scheme enables $epsilon$-DP users to employ looser DP guarantees when training their model to limit the success of any MI adversary.
- Score: 28.64031194463754
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Differential Privacy (DP) is the de facto standard for reasoning about the
privacy guarantees of a training algorithm. Despite the empirical observation
that DP reduces the vulnerability of models to existing membership inference
(MI) attacks, a theoretical underpinning as to why this is the case is largely
missing in the literature. In practice, this means that models need to be
trained with DP guarantees that greatly decrease their accuracy. In this paper,
we provide a tighter bound on the accuracy of any MI adversary when a training
algorithm provides $\epsilon$-DP. Our bound informs the design of a novel
privacy amplification scheme, where an effective training set is sub-sampled
from a larger set prior to the beginning of training, to greatly reduce the
bound on MI accuracy. As a result, our scheme enables $\epsilon$-DP users to
employ looser DP guarantees when training their model to limit the success of
any MI adversary; this ensures that the model's accuracy is less impacted by
the privacy guarantee. Finally, we discuss implications of our MI bound on the
field of machine unlearning.
Related papers
- Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning [59.29849532966454]
We propose PseudoProbability Unlearning (PPU), a novel method that enables models to forget data to adhere to privacy-preserving manner.
Our method achieves over 20% improvements in forgetting error compared to the state-of-the-art.
arXiv Detail & Related papers (2024-11-04T21:27:06Z) - Overconfidence is a Dangerous Thing: Mitigating Membership Inference
Attacks by Enforcing Less Confident Prediction [2.2336243882030025]
Machine learning models are vulnerable to membership inference attacks (MIAs)
This work proposes a defense technique, HAMP, that can achieve both strong membership privacy and high accuracy, without requiring extra data.
arXiv Detail & Related papers (2023-07-04T09:50:33Z) - Discriminative Adversarial Privacy: Balancing Accuracy and Membership
Privacy in Neural Networks [7.0895962209555465]
Discriminative Adversarial Privacy (DAP) is a learning technique designed to achieve a balance between model performance, speed, and privacy.
DAP relies on adversarial training based on a novel loss function able to minimise the prediction error while maximising the MIA's error.
In addition, we introduce a novel metric named Accuracy Over Privacy (AOP) to capture the performance-privacy trade-off.
arXiv Detail & Related papers (2023-06-05T17:25:45Z) - Provable Membership Inference Privacy [31.08016816475564]
Differential privacy (DP) has emerged as one canonical standard for provable privacy.
We propose a novel privacy notion, membership inference privacy (MIP), to address these challenges.
We show MIP can be achieved using less amount of randomness compared to the amount required for guaranteeing DP, leading to a smaller drop in utility.
arXiv Detail & Related papers (2022-11-12T06:13:00Z) - RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
We propose a novel training framework based on a relaxed loss with a more achievable learning target.
RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead.
Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
arXiv Detail & Related papers (2022-07-12T19:34:47Z) - Large Scale Transfer Learning for Differentially Private Image
Classification [51.10365553035979]
Differential Privacy (DP) provides a formal framework for training machine learning models with individual example level privacy.
Private training using DP-SGD protects against leakage by injecting noise into individual example gradients.
While this result is quite appealing, the computational cost of training large-scale models with DP-SGD is substantially higher than non-private training.
arXiv Detail & Related papers (2022-05-06T01:22:20Z) - Optimal Membership Inference Bounds for Adaptive Composition of Sampled
Gaussian Mechanisms [93.44378960676897]
Given a trained model and a data sample, membership-inference (MI) attacks predict whether the sample was in the model's training set.
A common countermeasure against MI attacks is to utilize differential privacy (DP) during model training to mask the presence of individual examples.
In this paper, we derive bounds for the textitadvantage of an adversary mounting a MI attack, and demonstrate tightness for the widely-used Gaussian mechanism.
arXiv Detail & Related papers (2022-04-12T22:36:56Z) - Federated Learning with Sparsified Model Perturbation: Improving
Accuracy under Client-Level Differential Privacy [27.243322019117144]
Federated learning (FL) enables distributed clients to collaboratively learn a shared statistical model.
sensitive information about the training data can still be inferred from model updates shared in FL.
Differential privacy (DP) is the state-of-the-art technique to defend against those attacks.
This paper develops a novel FL scheme named Fed-SMP that provides client-level DP guarantee while maintaining high model accuracy.
arXiv Detail & Related papers (2022-02-15T04:05:42Z) - DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with
Differentially Private Data Augmentations [54.960853673256]
We show that strong data augmentations, such as mixup and random additive noise, nullify poison attacks while enduring only a small accuracy trade-off.
A rigorous analysis of DP-InstaHide shows that mixup does indeed have privacy advantages, and that training with k-way mixup provably yields at least k times stronger DP guarantees than a naive DP mechanism.
arXiv Detail & Related papers (2021-03-02T23:07:31Z) - How Does Data Augmentation Affect Privacy in Machine Learning? [94.52721115660626]
We propose new MI attacks to utilize the information of augmented data.
We establish the optimal membership inference when the model is trained with augmented data.
arXiv Detail & Related papers (2020-07-21T02:21:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.