Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference

• URL: http://arxiv.org/abs/2407.17663v1
• Date: Wed, 24 Jul 2024 22:16:37 GMT
• Title: Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference
• Authors: Catherine Huang, Martin Pawelczyk, Himabindu Lakkaraju,
• Abstract summary: We present two new membership inference attacks based on feature attribution explanations. We find that optimized differentially private fine-tuning substantially diminishes the success of the aforementioned attacks.
• Score: 26.596877194118278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predictive machine learning models are becoming increasingly deployed in high-stakes contexts involving sensitive personal data; in these contexts, there is a trade-off between model explainability and data privacy. In this work, we push the boundaries of this trade-off: with a focus on foundation models for image classification fine-tuning, we reveal unforeseen privacy risks of post-hoc model explanations and subsequently offer mitigation strategies for such risks. First, we construct VAR-LRT and L1/L2-LRT, two new membership inference attacks based on feature attribution explanations that are significantly more successful than existing explanation-leveraging attacks, particularly in the low false-positive rate regime that allows an adversary to identify specific training set members with confidence. Second, we find empirically that optimized differentially private fine-tuning substantially diminishes the success of the aforementioned attacks, while maintaining high model accuracy. We carry out a systematic empirical investigation of our 2 new attacks with 5 vision transformer architectures, 5 benchmark datasets, 4 state-of-the-art post-hoc explanation methods, and 4 privacy strength settings.

Related papers

• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Visual Privacy Auditing with Diffusion Models [52.866433097406656]
  We propose a reconstruction attack based on diffusion models (DMs) that assumes adversary access to real-world image priors. We show that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and (3) DMs can serve as effective auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning [77.27443885999404]
  Federated Learning (FL) is a setting for training machine learning models in distributed environments. We propose a novel method, CANIFE, that uses carefully crafted samples by a strong adversary to evaluate the empirical privacy of a training round.
  arXiv  Detail & Related papers  (2022-10-06T13:30:16Z)
• Generative Models with Information-Theoretic Protection Against Membership Inference Attacks [6.840474688871695]
  Deep generative models, such as Generative Adversarial Networks (GANs), synthesize diverse high-fidelity data samples. GANs may disclose private information from the data they are trained on, making them susceptible to adversarial attacks. We propose an information theoretically motivated regularization term that prevents the generative model from overfitting to training data and encourages generalizability.
  arXiv  Detail & Related papers  (2022-05-31T19:29:55Z)
• Enhanced Membership Inference Attacks against Machine Learning Models [9.26208227402571]
  Membership inference attacks are used to quantify the private information that a model leaks about the individual data points in its training set. We derive new attack algorithms that can achieve a high AUC score while also highlighting the different factors that affect their performance. Our algorithms capture a very precise approximation of privacy loss in models, and can be used as a tool to perform an accurate and informed estimation of privacy risk in machine learning models.
  arXiv  Detail & Related papers  (2021-11-18T13:31:22Z)
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [64.03398193325572]
  Inference attacks against Machine Learning (ML) models allow adversaries to learn about training data, model parameters, etc. We concentrate on four attacks - namely, membership inference, model inversion, attribute inference, and model stealing. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models.
  arXiv  Detail & Related papers  (2021-02-04T11:35:13Z)
• Knowledge-Enriched Distributional Model Inversion Attacks [49.43828150561947]
  Model inversion (MI) attacks are aimed at reconstructing training data from model parameters. We present a novel inversion-specific GAN that can better distill knowledge useful for performing attacks on private models from public data. Our experiments show that the combination of these techniques can significantly boost the success rate of the state-of-the-art MI attacks by 150%.
  arXiv  Detail & Related papers  (2020-10-08T16:20:48Z)
• Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries [74.59376038272661]
  We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model. We show that a victim model that only publishes the labels is still susceptible to sampling attacks and the adversary can recover up to 100% of its performance. For defense, we choose differential privacy in the form of gradient perturbation during the training of the victim model as well as output perturbation at prediction time.
  arXiv  Detail & Related papers  (2020-09-01T12:54:54Z)
• Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks [31.34410250008759]
  This paper measures the trade-off between model accuracy and privacy losses incurred by reconstruction, tracing and membership attacks. Experiments show that model accuracies are improved on average by 5-20% compared with baseline mechanisms.
  arXiv  Detail & Related papers  (2020-06-20T15:48:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.