Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models

• URL: http://arxiv.org/abs/2409.11219v2
• Date: Tue, 8 Oct 2024 03:59:06 GMT
• Title: Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models
• Authors: Tianqi Chen, Shujian Zhang, Mingyuan Zhou,
• Abstract summary: Machine unlearning (MU) is a crucial foundation for developing safe, secure, and trustworthy GenAI models. Traditional MU methods often rely on stringent assumptions and require access to real data. This paper introduces Score Forgetting Distillation (SFD), an innovative MU approach that promotes the forgetting of undesirable information in diffusion models.
• Score: 63.43422118066493
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The machine learning community is increasingly recognizing the importance of fostering trust and safety in modern generative AI (GenAI) models. We posit machine unlearning (MU) as a crucial foundation for developing safe, secure, and trustworthy GenAI models. Traditional MU methods often rely on stringent assumptions and require access to real data. This paper introduces Score Forgetting Distillation (SFD), an innovative MU approach that promotes the forgetting of undesirable information in diffusion models by aligning the conditional scores of "unsafe" classes or concepts with those of "safe" ones. To eliminate the need for real data, our SFD framework incorporates a score-based MU loss into the score distillation objective of a pretrained diffusion model. This serves as a regularization term that preserves desired generation capabilities while enabling the production of synthetic data through a one-step generator. Our experiments on pretrained label-conditional and text-to-image diffusion models demonstrate that our method effectively accelerates the forgetting of target classes or concepts during generation, while preserving the quality of other classes or concepts. This unlearned and distilled diffusion not only pioneers a novel concept in MU but also accelerates the generation speed of diffusion models. Our experiments and studies on a range of diffusion models and datasets confirm that our approach is generalizable, effective, and advantageous for MU in diffusion models. (Warning: This paper contains sexually explicit imagery, discussions of pornography, racially-charged terminology, and other content that some readers may find disturbing, distressing, and/or offensive.)

Related papers

• Diffusion Imitation from Observation [4.205946699819021]
  adversarial imitation learning approaches learn a generator agent policy to produce state transitions that are indistinguishable to a discriminator. Motivated by the recent success of diffusion models in generative modeling, we propose to integrate a diffusion model into the adversarial imitation learning from observation framework.
  arXiv  Detail & Related papers  (2024-10-07T18:49:55Z)
• Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Models [12.386823277082746]
  Membership inference attacks (MIAs) on diffusion models have emerged as potential evidence of unauthorized data usage. Our study delves into the evaluation of state-of-the-art MIAs on diffusion models and reveals critical flaws and overly optimistic performance estimates. We introduce CopyMark, a more realistic MIA benchmark that distinguishes itself through the support for pre-trained diffusion models, unbiased datasets, and fair evaluation pipelines.
  arXiv  Detail & Related papers  (2024-10-04T17:46:06Z)
• Unlearning or Concealment? A Critical Analysis and Evaluation Metrics for Unlearning in Diffusion Models [7.9993879763024065]
  We show that the objective functions used for unlearning in the existing methods lead to decoupling of the targeted concepts for the corresponding prompts. The ineffectiveness of current methods stems primarily from their narrow focus on reducing generation probabilities for specific prompt sets. We introduce two new evaluation metrics: Concept Retrieval Score (CRS) and Concept Confidence Score (CCS)
  arXiv  Detail & Related papers  (2024-09-09T14:38:31Z)
• Adversarial Robustification via Text-to-Image Diffusion Models [56.37291240867549]
  Adrial robustness has been conventionally believed as a challenging property to encode for neural networks. We develop a scalable and model-agnostic solution to achieve adversarial robustness without using any data.
  arXiv  Detail & Related papers  (2024-07-26T10:49:14Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Latent Diffusion Counterfactual Explanations [28.574246724214962]
  We introduce Latent Diffusion Counterfactual Explanations (LDCE) LDCE harnesses the capabilities of recent class- or text-conditional foundation latent diffusion models to expedite counterfactual generation. We show how LDCE can provide insights into model errors, enhancing our understanding of black-box model behavior.
  arXiv  Detail & Related papers  (2023-10-10T14:42:34Z)
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models [77.83923746319498]
  We propose a framework called Diff-Instruct to instruct the training of arbitrary generative models. We show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. Experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models.
  arXiv  Detail & Related papers  (2023-05-29T04:22:57Z)
• Your Diffusion Model is Secretly a Zero-Shot Classifier [90.40799216880342]
  We show that density estimates from large-scale text-to-image diffusion models can be leveraged to perform zero-shot classification. Our generative approach to classification attains strong results on a variety of benchmarks. Our results are a step toward using generative over discriminative models for downstream tasks.
  arXiv  Detail & Related papers  (2023-03-28T17:59:56Z)
• DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models [81.84866217721361]
  DiffusionBERT is a new generative masked language model based on discrete diffusion models. We propose a new noise schedule for the forward diffusion process that controls the degree of noise added at each step. Experiments on unconditional text generation demonstrate that DiffusionBERT achieves significant improvement over existing diffusion models for text.
  arXiv  Detail & Related papers  (2022-11-28T03:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.