Membership Inference Attacks against Language Models via Neighbourhood Comparison

• URL: http://arxiv.org/abs/2305.18462v2
• Date: Mon, 7 Aug 2023 06:32:56 GMT
• Title: Membership Inference Attacks against Language Models via Neighbourhood Comparison
• Authors: Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Sch\"olkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick
• Abstract summary: Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not. Recent work has demonstrated that reference-based attacks which compare model scores to those obtained from a reference model trained on similar data can substantially improve the performance of MIAs. We investigate their performance in more realistic scenarios and find that they are highly fragile in relation to the data distribution used to train reference models.
• Score: 45.086816556309266
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not, and are widely used for assessing the privacy risks of language models. Most existing attacks rely on the observation that models tend to assign higher probabilities to their training samples than non-training points. However, simple thresholding of the model score in isolation tends to lead to high false-positive rates as it does not account for the intrinsic complexity of a sample. Recent work has demonstrated that reference-based attacks which compare model scores to those obtained from a reference model trained on similar data can substantially improve the performance of MIAs. However, in order to train reference models, attacks of this kind make the strong and arguably unrealistic assumption that an adversary has access to samples closely resembling the original training data. Therefore, we investigate their performance in more realistic scenarios and find that they are highly fragile in relation to the data distribution used to train reference models. To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution. We show that, in addition to being competitive with reference-based attacks that have perfect knowledge about the training data distribution, our attack clearly outperforms existing reference-free attacks as well as reference-based attacks with imperfect knowledge, which demonstrates the need for a reevaluation of the threat model of adversarial attacks.

Related papers

• When Fairness Meets Privacy: Exploring Privacy Threats in Fair Binary Classifiers via Membership Inference Attacks [17.243744418309593]
  We propose an efficient MIA method against fairness-enhanced models based on fairness discrepancy results. We also explore potential strategies for mitigating privacy leakages.
  arXiv  Detail & Related papers  (2023-11-07T10:28:17Z)
• Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks [65.21536453075275]
  We focus on the summarization task and investigate the membership inference (MI) attack. We exploit text similarity and the model's resistance to document modifications as potential MI signals. We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2023-10-20T05:44:39Z)
• OMG-ATTACK: Self-Supervised On-Manifold Generation of Transferable Evasion Attacks [17.584752814352502]
  Evasion Attacks (EA) are used to test the robustness of trained neural networks by distorting input data. We introduce a self-supervised, computationally economical method for generating adversarial examples. Our experiments consistently demonstrate the method is effective across various models, unseen data categories, and even defended models.
  arXiv  Detail & Related papers  (2023-10-05T17:34:47Z)
• Towards Poisoning Fair Representations [26.47681999979761]
  This work proposes the first data poisoning framework attacking fair representation learning methods. We induce the model to output unfair representations that contain as much demographic information as possible by injecting carefully crafted poisoning samples into the training data. Experiments on benchmark fairness datasets and state-of-the-art fair representation learning models demonstrate the superiority of our attack.
  arXiv  Detail & Related papers  (2023-09-28T14:51:20Z)
• Boosting Model Inversion Attacks with Adversarial Examples [26.904051413441316]
  We propose a new training paradigm for a learning-based model inversion attack that can achieve higher attack accuracy in a black-box setting. First, we regularize the training process of the attack model with an added semantic loss function. Second, we inject adversarial examples into the training data to increase the diversity of the class-related parts.
  arXiv  Detail & Related papers  (2023-06-24T13:40:58Z)
• A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks [72.7373468905418]
  We develop an open-source toolkit OpenBackdoor to foster the implementations and evaluations of textual backdoor learning. We also propose CUBE, a simple yet strong clustering-based defense baseline.
  arXiv  Detail & Related papers  (2022-06-17T02:29:23Z)
• Enhanced Membership Inference Attacks against Machine Learning Models [9.26208227402571]
  Membership inference attacks are used to quantify the private information that a model leaks about the individual data points in its training set. We derive new attack algorithms that can achieve a high AUC score while also highlighting the different factors that affect their performance. Our algorithms capture a very precise approximation of privacy loss in models, and can be used as a tool to perform an accurate and informed estimation of privacy risk in machine learning models.
  arXiv  Detail & Related papers  (2021-11-18T13:31:22Z)
• Delving into Data: Effectively Substitute Training for Black-box Attack [84.85798059317963]
  We propose a novel perspective substitute training that focuses on designing the distribution of data used in the knowledge stealing process. The combination of these two modules can further boost the consistency of the substitute model and target model, which greatly improves the effectiveness of adversarial attack.
  arXiv  Detail & Related papers  (2021-04-26T07:26:29Z)
• Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations [81.82518920087175]
  Adversarial attacking aims to fool deep neural networks with adversarial examples. We propose a reinforcement learning based attack model, which can learn from attack history and launch attacks more efficiently.
  arXiv  Detail & Related papers  (2020-09-19T09:12:24Z)
• Boosting Black-Box Attack with Partially Transferred Conditional Adversarial Distribution [83.02632136860976]
  We study black-box adversarial attacks against deep neural networks (DNNs) We develop a novel mechanism of adversarial transferability, which is robust to the surrogate biases. Experiments on benchmark datasets and attacking against real-world API demonstrate the superior attack performance of the proposed method.
  arXiv  Detail & Related papers  (2020-06-15T16:45:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.