PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining

• URL: http://arxiv.org/abs/2402.09477v1
• Date: Mon, 12 Feb 2024 22:56:07 GMT
• Title: PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining
• Authors: Mishaal Kazmi, Hadrien Lautraite, Alireza Akbari, Mauricio Soroco, Qiaoyue Tang, Tao Wang, S\'ebastien Gambs, Mathias L\'ecuyer
• Abstract summary: We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members" This scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale ML models without control of the training process or model re-training.
• Score: 2.6068944905108227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members". This scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale ML models without control of the training process or model re-training and only requires access to a subset of the training data. To demonstrate its applicability, we evaluate our auditing scheme across multiple ML domains, ranging from image and tabular data classification to large-scale language models.

Related papers

• Membership Inference Attack Should Move On to Distributional Statistics for Distilled Generative Models [31.834967019893227]
  Membership inference attacks (MIAs) determine whether certain data instances were used to train a model. This paper reveals an oversight in existing MIAs against emphdistilled generative models We introduce a emphset-based MIA framework that measures emphrelative distributional discrepancies between student-generated dataemphsets and potential member/non-member dataemphsets
  arXiv  Detail & Related papers  (2025-02-05T08:11:23Z)
• Self-Comparison for Dataset-Level Membership Inference in Large (Vision-)Language Models [73.94175015918059]
  We propose a dataset-level membership inference method based on Self-Comparison. Our method does not require access to ground-truth member data or non-member data in identical distribution.
  arXiv  Detail & Related papers  (2024-10-16T23:05:59Z)
• A General Framework for Data-Use Auditing of ML Models [47.369572284751285]
  We propose a general method to audit an ML model for the use of a data-owner's data in training. We show the effectiveness of our proposed framework by applying it to audit data use in two types of ML models.
  arXiv  Detail & Related papers  (2024-07-21T09:32:34Z)
• A Method to Facilitate Membership Inference Attacks in Deep Learning Models [5.724311218570013]
  We demonstrate a new form of membership inference attack that is strictly more powerful than prior art. Our attack empowers the adversary to reliably de-identify all the training samples. We show that the models can effectively disguise the amplified membership leakage under common membership privacy auditing.
  arXiv  Detail & Related papers  (2024-07-02T03:33:42Z)
• Do Membership Inference Attacks Work on Large Language Models? [141.2019867466968]
  Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model's training data. We perform a large-scale evaluation of MIAs over a suite of language models trained on the Pile, ranging from 160M to 12B parameters. We find that MIAs barely outperform random guessing for most settings across varying LLM sizes and domains.
  arXiv  Detail & Related papers  (2024-02-12T17:52:05Z)
• Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks [65.21536453075275]
  We focus on the summarization task and investigate the membership inference (MI) attack. We exploit text similarity and the model's resistance to document modifications as potential MI signals. We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2023-10-20T05:44:39Z)
• AI Model Disgorgement: Methods and Choices [127.54319351058167]
  We introduce a taxonomy of possible disgorgement methods that are applicable to modern machine learning systems. We investigate the meaning of "removing the effects" of data in the trained model in a way that does not require retraining from scratch.
  arXiv  Detail & Related papers  (2023-04-07T08:50:18Z)
• Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
  Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. This creates a barrier to fusing knowledge across individual models to yield a better single model. We propose a dataless knowledge fusion method that merges models in their parameter space.
  arXiv  Detail & Related papers  (2022-12-19T20:46:43Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)
• Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture [44.2351146468898]
  Membership inference attacks are a key measure to evaluate privacy leakage in machine learning (ML) models. We propose a new framework to train privacy-preserving models that induce similar behavior on member and non-member inputs. We show that SELENA presents a superior trade-off between membership privacy and utility compared to the state of the art.
  arXiv  Detail & Related papers  (2021-10-15T19:22:52Z)
• Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning [27.494206948563885]
  We present the first systematic evaluation of membership inference attacks against transfer learning models. Experiments on four real-world image datasets show that membership inference can achieve effective performance. Our results shed light on the severity of membership risks stemming from machine learning models in practice.
  arXiv  Detail & Related papers  (2020-09-10T14:14:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.