PANORAMIA: Privacy Auditing of Machine Learning Models without
Retraining
- URL: http://arxiv.org/abs/2402.09477v1
- Date: Mon, 12 Feb 2024 22:56:07 GMT
- Title: PANORAMIA: Privacy Auditing of Machine Learning Models without
Retraining
- Authors: Mishaal Kazmi, Hadrien Lautraite, Alireza Akbari, Mauricio Soroco,
Qiaoyue Tang, Tao Wang, S\'ebastien Gambs, Mathias L\'ecuyer
- Abstract summary: We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members"
This scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale ML models without control of the training process or model re-training.
- Score: 2.6068944905108227
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce a privacy auditing scheme for ML models that relies on
membership inference attacks using generated data as "non-members". This
scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale
ML models without control of the training process or model re-training and only
requires access to a subset of the training data. To demonstrate its
applicability, we evaluate our auditing scheme across multiple ML domains,
ranging from image and tabular data classification to large-scale language
models.
Related papers
- A General Framework for Data-Use Auditing of ML Models [47.369572284751285]
We propose a general method to audit an ML model for the use of a data-owner's data in training.
We show the effectiveness of our proposed framework by applying it to audit data use in two types of ML models.
arXiv Detail & Related papers (2024-07-21T09:32:34Z) - A Method to Facilitate Membership Inference Attacks in Deep Learning Models [5.724311218570013]
We demonstrate a new form of membership inference attack that is strictly more powerful than prior art.
Our attack empowers the adversary to reliably de-identify all the training samples.
We show that the models can effectively disguise the amplified membership leakage under common membership privacy auditing.
arXiv Detail & Related papers (2024-07-02T03:33:42Z) - Data Shapley in One Training Run [88.59484417202454]
Data Shapley provides a principled framework for attributing data's contribution within machine learning contexts.
Existing approaches require re-training models on different data subsets, which is computationally intensive.
This paper introduces In-Run Data Shapley, which addresses these limitations by offering scalable data attribution for a target model of interest.
arXiv Detail & Related papers (2024-06-16T17:09:24Z) - Do Membership Inference Attacks Work on Large Language Models? [145.90022632726883]
Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model's training data.
We perform a large-scale evaluation of MIAs over a suite of language models trained on the Pile, ranging from 160M to 12B parameters.
We find that MIAs barely outperform random guessing for most settings across varying LLM sizes and domains.
arXiv Detail & Related papers (2024-02-12T17:52:05Z) - Assessing Privacy Risks in Language Models: A Case Study on
Summarization Tasks [65.21536453075275]
We focus on the summarization task and investigate the membership inference (MI) attack.
We exploit text similarity and the model's resistance to document modifications as potential MI signals.
We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
arXiv Detail & Related papers (2023-10-20T05:44:39Z) - Privacy Side Channels in Machine Learning Systems [87.53240071195168]
We introduce privacy side channels: attacks that exploit system-level components to extract private information.
For example, we show that deduplicating training data before applying differentially-private training creates a side-channel that completely invalidates any provable privacy guarantees.
We further show that systems which block language models from regenerating training data can be exploited to exfiltrate private keys contained in the training set.
arXiv Detail & Related papers (2023-09-11T16:49:05Z) - Accuracy Improvement in Differentially Private Logistic Regression: A
Pre-training Approach [4.297070083645049]
This paper aims to boost the accuracy of a DP logistic regression (LR) model via a pre-training module.
In the numerical results, we show that adding a pre-training module significantly improves the accuracy of the DP-LR model.
arXiv Detail & Related papers (2023-07-25T19:07:03Z) - Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models.
This creates a barrier to fusing knowledge across individual models to yield a better single model.
We propose a dataless knowledge fusion method that merges models in their parameter space.
arXiv Detail & Related papers (2022-12-19T20:46:43Z) - Privacy Analysis of Deep Learning in the Wild: Membership Inference
Attacks against Transfer Learning [27.494206948563885]
We present the first systematic evaluation of membership inference attacks against transfer learning models.
Experiments on four real-world image datasets show that membership inference can achieve effective performance.
Our results shed light on the severity of membership risks stemming from machine learning models in practice.
arXiv Detail & Related papers (2020-09-10T14:14:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.