Visual Privacy Auditing with Diffusion Models

• URL: http://arxiv.org/abs/2403.07588v2
• Date: Sun, 09 Mar 2025 08:21:00 GMT
• Title: Visual Privacy Auditing with Diffusion Models
• Authors: Kristian Schwethelm, Johannes Kaiser, Moritz Knolle, Sarah Lockfisch, Daniel Rueckert, Alexander Ziller,
• Abstract summary: We introduce a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors.<n>We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and DMs can serve as auditing tools for visualizing privacy leakage.
• Score: 47.0700328585184
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data reconstruction attacks on machine learning models pose a substantial threat to privacy, potentially leaking sensitive information. Although defending against such attacks using differential privacy (DP) provides theoretical guarantees, determining appropriate DP parameters remains challenging. Current formal guarantees on the success of data reconstruction suffer from overly stringent assumptions regarding adversary knowledge about the target data, particularly in the image domain, raising questions about their real-world applicability. In this work, we empirically investigate this discrepancy by introducing a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors and specifically targets the DP defense. We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and (3) DMs can serve as heuristic auditing tools for visualizing privacy leakage.

Related papers

• Model Inversion Attacks: A Survey of Approaches and Countermeasures [59.986922963781]
  Recently, a new type of privacy attack, the model inversion attacks (MIAs), aims to extract sensitive features of private data for training. Despite the significance, there is a lack of systematic studies that provide a comprehensive overview and deeper insights into MIAs. This survey aims to summarize up-to-date MIA methods in both attacks and defenses.
  arXiv  Detail & Related papers  (2024-11-15T08:09:28Z)
• Demystifying Trajectory Recovery From Ash: An Open-Source Evaluation and Enhancement [5.409124675229009]
  This study reimplements the trajectory recovery attack from scratch and evaluates it on two open-source datasets. Results confirm that privacy leakage still exists despite common anonymisation and aggregation methods. We propose a stronger attack by designing a series of enhancements to the baseline attack.
  arXiv  Detail & Related papers  (2024-09-23T01:06:41Z)
• Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference [26.596877194118278]
  We present two new membership inference attacks based on feature attribution explanations. We find that optimized differentially private fine-tuning substantially diminishes the success of the aforementioned attacks.
  arXiv  Detail & Related papers  (2024-07-24T22:16:37Z)
• Information Leakage from Embedding in Large Language Models [5.475800773759642]
  This study aims to investigate the potential for privacy invasion through input reconstruction attacks. We first propose two base methods to reconstruct original texts from a model's hidden states. We then present Embed Parrot, a Transformer-based method, to reconstruct input from embeddings in deep layers.
  arXiv  Detail & Related papers  (2024-05-20T09:52:31Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Bounding Reconstruction Attack Success of Adversaries Without Data Priors [53.41619942066895]
  Reconstruction attacks on machine learning (ML) models pose a strong risk of leakage of sensitive data. In this work, we provide formal upper bounds on reconstruction success under realistic adversarial settings.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Reconciling AI Performance and Data Reconstruction Resilience for Medical Imaging [52.578054703818125]
  Artificial Intelligence (AI) models are vulnerable to information leakage of their training data, which can be highly sensitive. Differential Privacy (DP) aims to circumvent these susceptibilities by setting a quantifiable privacy budget. We show that using very large privacy budgets can render reconstruction attacks impossible, while drops in performance are negligible.
  arXiv  Detail & Related papers  (2023-12-05T12:21:30Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy [62.16582309504159]
  We develop a systematic analysis of membership inference attacks on diffusion models and propose novel attack methods tailored to each attack scenario. Our approach exploits easily obtainable quantities and is highly effective, achieving near-perfect attack performance (>0.9 AUCROC) in realistic scenarios.
  arXiv  Detail & Related papers  (2023-02-15T17:37:49Z)
• Defending against Reconstruction Attacks with R\'enyi Differential Privacy [72.1188520352079]
  Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget. We show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature.
  arXiv  Detail & Related papers  (2022-02-15T18:09:30Z)
• Federated Test-Time Adaptive Face Presentation Attack Detection with Dual-Phase Privacy Preservation [100.69458267888962]
  Face presentation attack detection (fPAD) plays a critical role in the modern face recognition pipeline. Due to legal and privacy issues, training data (real face images and spoof images) are not allowed to be directly shared between different data sources. We propose a Federated Test-Time Adaptive Face Presentation Attack Detection with Dual-Phase Privacy Preservation framework.
  arXiv  Detail & Related papers  (2021-10-25T02:51:05Z)
• Federated Deep Learning with Bayesian Privacy [28.99404058773532]
  Federated learning (FL) aims to protect data privacy by cooperatively learning a model without sharing private data among users. Homomorphic encryption (HE) based methods provide secure privacy protections but suffer from extremely high computational and communication overheads. Deep learning with Differential Privacy (DP) was implemented as a practical learning algorithm at a manageable cost in complexity.
  arXiv  Detail & Related papers  (2021-09-27T12:48:40Z)
• Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks [31.34410250008759]
  This paper measures the trade-off between model accuracy and privacy losses incurred by reconstruction, tracing and membership attacks. Experiments show that model accuracies are improved on average by 5-20% compared with baseline mechanisms.
  arXiv  Detail & Related papers  (2020-06-20T15:48:57Z)
• Mitigating Query-Flooding Parameter Duplication Attack on Regression Models with High-Dimensional Gaussian Mechanism [12.017509695576377]
  Differential privacy (DP) has been considered a promising technique to mitigate this attack. We show that the adversary can launch a query-flooding parameter duplication (QPD) attack to infer the model information. We propose a novel High-Dimensional Gaussian (HDG) mechanism to prevent unauthorized information disclosure.
  arXiv  Detail & Related papers  (2020-02-06T01:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.