Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk

• URL: http://arxiv.org/abs/2403.09450v2
• Date: Mon, 22 Apr 2024 16:48:39 GMT
• Title: Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk
• Authors: Zhangheng Li, Junyuan Hong, Bo Li, Zhangyang Wang,
• Abstract summary: We reveal a new privacy risk, Shake-to-Leak (S2L), that fine-tuning the pre-trained models with manipulated data can amplify the existing privacy risks. In the worst case, S2L can amplify the state-of-the-art membership inference attack (MIA) on diffusion models by $5.4%$ AUC. This discovery underscores that the privacy risk with diffusion models is even more severe than previously recognized.
• Score: 60.36852134501251
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information. In this paper, we reveal a new risk, Shake-to-Leak (S2L), that fine-tuning the pre-trained models with manipulated data can amplify the existing privacy risks. We demonstrate that S2L could occur in various standard fine-tuning strategies for diffusion models, including concept-injection methods (DreamBooth and Textual Inversion) and parameter-efficient methods (LoRA and Hypernetwork), as well as their combinations. In the worst case, S2L can amplify the state-of-the-art membership inference attack (MIA) on diffusion models by $5.4\%$ (absolute difference) AUC and can increase extracted private samples from almost $0$ samples to $15.8$ samples on average per target domain. This discovery underscores that the privacy risk with diffusion models is even more severe than previously recognized. Codes are available at https://github.com/VITA-Group/Shake-to-Leak.

Related papers

• Gradient Inversion of Federated Diffusion Models [4.1355611383748005]
  Diffusion models are becoming defector generative models, which generate exceptionally high-resolution image data. In this paper, we study the privacy risk of gradient inversion attacks. We propose a triple-optimization GIDM+ that coordinates the optimization of the unknown data.
  arXiv  Detail & Related papers  (2024-05-30T18:00:03Z)
• Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data [74.2507346810066]
  Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. We present the first framework for training diffusion models that provably sample from the uncorrupted distribution given only noisy training data.
  arXiv  Detail & Related papers  (2024-03-20T14:22:12Z)
• PriSampler: Mitigating Property Inference of Diffusion Models [6.5990719141691825]
  This work systematically presents the first privacy study about property inference attacks against diffusion models. We propose a new model-agnostic plug-in method PriSampler to infer the risks of the property inference of diffusion models.
  arXiv  Detail & Related papers  (2023-06-08T14:05:06Z)
• Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Models [11.659461421660613]
  We introduce Privacy Distillation, a framework that allows a text-to-image generative model to teach another model without exposing it to identifiable data. Our solution consists of (1) training a first diffusion model on real data (2) generating a synthetic dataset using this model and filtering it to exclude images with a re-identifiability risk (3) training a second diffusion model on the filtered synthetic data only.
  arXiv  Detail & Related papers  (2023-06-02T07:44:00Z)
• An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization [58.88327181933151]
  In this paper, we propose an efficient query-based membership inference attack (MIA) Experimental results indicate that the proposed method can achieve competitive performance with only two queries on both discrete-time and continuous-time diffusion models. To the best of our knowledge, this work is the first to study the robustness of diffusion models to MIA in the text-to-speech task.
  arXiv  Detail & Related papers  (2023-05-26T16:38:48Z)
• Are Diffusion Models Vulnerable to Membership Inference Attacks? [26.35177414594631]
  Diffusion-based generative models have shown great potential for image synthesis, but there is a lack of research on the security and privacy risks they may pose. We investigate the vulnerability of diffusion models to Membership Inference Attacks (MIAs), a common privacy concern. We propose Step-wise Error Comparing Membership Inference (SecMI), a query-based MIA that infers memberships by assessing the matching of forward process posterior estimation at each timestep.
  arXiv  Detail & Related papers  (2023-02-02T18:43:16Z)
• Extracting Training Data from Diffusion Models [77.11719063152027]
  We show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models. We train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy.
  arXiv  Detail & Related papers  (2023-01-30T18:53:09Z)
• How to Backdoor Diffusion Models? [74.43215520371506]
  This paper presents the first study on the robustness of diffusion models against backdoor attacks. We propose BadDiffusion, a novel attack framework that engineers compromised diffusion processes during model training for backdoor implantation. Our results call attention to potential risks and possible misuse of diffusion models.
  arXiv  Detail & Related papers  (2022-12-11T03:44:38Z)
• CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning [77.27443885999404]
  Federated Learning (FL) is a setting for training machine learning models in distributed environments. We propose a novel method, CANIFE, that uses carefully crafted samples by a strong adversary to evaluate the empirical privacy of a training round.
  arXiv  Detail & Related papers  (2022-10-06T13:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.