Visual Privacy Auditing with Diffusion Models
- URL: http://arxiv.org/abs/2403.07588v1
- Date: Tue, 12 Mar 2024 12:18:55 GMT
- Title: Visual Privacy Auditing with Diffusion Models
- Authors: Kristian Schwethelm, Johannes Kaiser, Moritz Knolle, Daniel Rueckert,
Georgios Kaissis, Alexander Ziller
- Abstract summary: We propose a reconstruction attack based on diffusion models (DMs) that assumes adversary access to real-world image priors.
We show that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and (3) DMs can serve as effective auditing tools for visualizing privacy leakage.
- Score: 52.866433097406656
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Image reconstruction attacks on machine learning models pose a significant
risk to privacy by potentially leaking sensitive information. Although
defending against such attacks using differential privacy (DP) has proven
effective, determining appropriate DP parameters remains challenging. Current
formal guarantees on data reconstruction success suffer from overly theoretical
assumptions regarding adversary knowledge about the target data, particularly
in the image domain. In this work, we empirically investigate this discrepancy
and find that the practicality of these assumptions strongly depends on the
domain shift between the data prior and the reconstruction target. We propose a
reconstruction attack based on diffusion models (DMs) that assumes adversary
access to real-world image priors and assess its implications on privacy
leakage under DP-SGD. We show that (1) real-world data priors significantly
influence reconstruction success, (2) current reconstruction bounds do not
model the risk posed by data priors well, and (3) DMs can serve as effective
auditing tools for visualizing privacy leakage.
Related papers
- Demystifying Trajectory Recovery From Ash: An Open-Source Evaluation and Enhancement [5.409124675229009]
This study reimplements the trajectory recovery attack from scratch and evaluates it on two open-source datasets.
Results confirm that privacy leakage still exists despite common anonymisation and aggregation methods.
We propose a stronger attack by designing a series of enhancements to the baseline attack.
arXiv Detail & Related papers (2024-09-23T01:06:41Z) - Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference [26.596877194118278]
We present two new membership inference attacks based on feature attribution explanations.
We find that optimized differentially private fine-tuning substantially diminishes the success of the aforementioned attacks.
arXiv Detail & Related papers (2024-07-24T22:16:37Z) - Information Leakage from Embedding in Large Language Models [5.475800773759642]
This study aims to investigate the potential for privacy invasion through input reconstruction attacks.
We first propose two base methods to reconstruct original texts from a model's hidden states.
We then present Embed Parrot, a Transformer-based method, to reconstruct input from embeddings in deep layers.
arXiv Detail & Related papers (2024-05-20T09:52:31Z) - Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
In this paper, we unveil a new vulnerability: the privacy backdoor attack.
When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model.
Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
arXiv Detail & Related papers (2024-04-01T16:50:54Z) - Bounding Reconstruction Attack Success of Adversaries Without Data
Priors [53.41619942066895]
Reconstruction attacks on machine learning (ML) models pose a strong risk of leakage of sensitive data.
In this work, we provide formal upper bounds on reconstruction success under realistic adversarial settings.
arXiv Detail & Related papers (2024-02-20T09:52:30Z) - Reconciling AI Performance and Data Reconstruction Resilience for
Medical Imaging [52.578054703818125]
Artificial Intelligence (AI) models are vulnerable to information leakage of their training data, which can be highly sensitive.
Differential Privacy (DP) aims to circumvent these susceptibilities by setting a quantifiable privacy budget.
We show that using very large privacy budgets can render reconstruction attacks impossible, while drops in performance are negligible.
arXiv Detail & Related papers (2023-12-05T12:21:30Z) - Avoid Adversarial Adaption in Federated Learning by Multi-Metric
Investigations [55.2480439325792]
Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources.
FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks.
We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously.
MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
arXiv Detail & Related papers (2023-06-06T11:44:42Z) - Data Forensics in Diffusion Models: A Systematic Analysis of Membership
Privacy [62.16582309504159]
We develop a systematic analysis of membership inference attacks on diffusion models and propose novel attack methods tailored to each attack scenario.
Our approach exploits easily obtainable quantities and is highly effective, achieving near-perfect attack performance (>0.9 AUCROC) in realistic scenarios.
arXiv Detail & Related papers (2023-02-15T17:37:49Z) - Federated Deep Learning with Bayesian Privacy [28.99404058773532]
Federated learning (FL) aims to protect data privacy by cooperatively learning a model without sharing private data among users.
Homomorphic encryption (HE) based methods provide secure privacy protections but suffer from extremely high computational and communication overheads.
Deep learning with Differential Privacy (DP) was implemented as a practical learning algorithm at a manageable cost in complexity.
arXiv Detail & Related papers (2021-09-27T12:48:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.