On the Discredibility of Membership Inference Attacks

• URL: http://arxiv.org/abs/2212.02701v2
• Date: Fri, 28 Apr 2023 16:49:01 GMT
• Title: On the Discredibility of Membership Inference Attacks
• Authors: Shahbaz Rezaei and Xin Liu
• Abstract summary: Membership inference attacks are proposed to determine if a sample was part of the training set or not. We show that MI models frequently misclassify neighboring nonmember samples of a member sample as members. We argue that current membership inference attacks can identify memorized subpopulations, but they cannot reliably identify which exact sample in the subpopulation was used during the training.
• Score: 11.172550334631921
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the wide-spread application of machine learning models, it has become critical to study the potential data leakage of models trained on sensitive data. Recently, various membership inference (MI) attacks are proposed to determine if a sample was part of the training set or not. The question is whether these attacks can be reliably used in practice. We show that MI models frequently misclassify neighboring nonmember samples of a member sample as members. In other words, they have a high false positive rate on the subpopulations of the exact member samples that they can identify. We then showcase a practical application of MI attacks where this issue has a real-world repercussion. Here, MI attacks are used by an external auditor (investigator) to show to a judge/jury that an auditee unlawfully used sensitive data. Due to the high false positive rate of MI attacks on member's subpopulations, auditee challenges the credibility of the auditor by revealing the performance of the MI attacks on these subpopulations. We argue that current membership inference attacks can identify memorized subpopulations, but they cannot reliably identify which exact sample in the subpopulation was used during the training.

Related papers

• Blind Baselines Beat Membership Inference Attacks for Foundation Models [24.010279957557252]
  Membership inference (MI) attacks try to determine if a data sample was used to train a machine learning model. For foundation models trained on unknown Web data, MI attacks can be used to detect copyrighted training materials, measure test set contamination, or audit machine unlearning. We show that evaluations of MI attacks for foundation models are flawed, because they sample members and non-members from different distributions.
  arXiv  Detail & Related papers  (2024-06-23T19:40:11Z)
• Do Membership Inference Attacks Work on Large Language Models? [141.2019867466968]
  Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model's training data. We perform a large-scale evaluation of MIAs over a suite of language models trained on the Pile, ranging from 160M to 12B parameters. We find that MIAs barely outperform random guessing for most settings across varying LLM sizes and domains.
  arXiv  Detail & Related papers  (2024-02-12T17:52:05Z)
• Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks [65.21536453075275]
  We focus on the summarization task and investigate the membership inference (MI) attack. We exploit text similarity and the model's resistance to document modifications as potential MI signals. We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2023-10-20T05:44:39Z)
• Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks [72.03945355787776]
  We advocate MDP, a lightweight, pluggable, and effective defense for PLMs as few-shot learners. We show analytically that MDP creates an interesting dilemma for the attacker to choose between attack effectiveness and detection evasiveness.
  arXiv  Detail & Related papers  (2023-09-23T04:41:55Z)
• Membership Inference Attacks against Synthetic Data through Overfitting Detection [84.02632160692995]
  We argue for a realistic MIA setting that assumes the attacker has some knowledge of the underlying data distribution. We propose DOMIAS, a density-based MIA model that aims to infer membership by targeting local overfitting of the generative model.
  arXiv  Detail & Related papers  (2023-02-24T11:27:39Z)
• Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries [53.222218035435006]
  We use adversarial tools to optimize for queries that are discriminative and diverse. Our improvements achieve significantly more accurate membership inference than existing methods.
  arXiv  Detail & Related papers  (2022-10-19T17:46:50Z)
• User-Level Membership Inference Attack against Metric Embedding Learning [8.414720636874106]
  Membership inference (MI) determines if a sample was part of a victim model training set. In this paper, we develop a user-level MI attack where the goal is to find if any sample from the target user has been used during training.
  arXiv  Detail & Related papers  (2022-03-04T00:49:42Z)
• Investigating Membership Inference Attacks under Data Dependencies [26.70764798408236]
  Training machine learning models on privacy-sensitive data has opened the door to new attacks that can have serious privacy implications. One such attack, the Membership Inference Attack (MIA), exposes whether or not a particular data point was used to train a model. We evaluate the defence under the restrictive assumption that all members of the training set, as well as non-members, are independent and identically distributed.
  arXiv  Detail & Related papers  (2020-10-23T00:16:46Z)
• Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries [74.59376038272661]
  We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model. We show that a victim model that only publishes the labels is still susceptible to sampling attacks and the adversary can recover up to 100% of its performance. For defense, we choose differential privacy in the form of gradient perturbation during the training of the victim model as well as output perturbation at prediction time.
  arXiv  Detail & Related papers  (2020-09-01T12:54:54Z)
• On the Difficulty of Membership Inference Attacks [11.172550334631921]
  Recent studies propose membership inference (MI) attacks on deep models. Despite their apparent success, these studies only report accuracy, precision, and recall of the positive class (member class) We show that the way the MI attack performance has been reported is often misleading because they suffer from high false positive rate or false alarm rate (FAR) that has not been reported.
  arXiv  Detail & Related papers  (2020-05-27T23:09:17Z)
• Membership Inference Attacks and Defenses in Classification Models [19.498313593713043]
  We study the membership inference (MI) attack against classifiers. We find that a model's vulnerability to MI attacks is tightly related to the generalization gap. We propose a defense against MI attacks that aims to close the gap by intentionally reducing the training accuracy.
  arXiv  Detail & Related papers  (2020-02-27T12:35:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.