EraseDiff: Erasing Data Influence in Diffusion Models

• URL: http://arxiv.org/abs/2401.05779v2
• Date: Mon, 5 Feb 2024 00:32:13 GMT
• Title: EraseDiff: Erasing Data Influence in Diffusion Models
• Authors: Jing Wu, Trung Le, Munawar Hayat, Mehrtash Harandi
• Abstract summary: We introduce an unlearning algorithm for diffusion models. We show that our algorithm can preserve the model utility, effectiveness, and efficiency while removing across the widely-used diffusion models.
• Score: 54.95692559939673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we introduce an unlearning algorithm for diffusion models. Our algorithm equips a diffusion model with a mechanism to mitigate the concerns related to data memorization. To achieve this, we formulate the unlearning problem as a constraint optimization problem, aiming to preserve the utility of the diffusion model on the remaining data and scrub the information associated with forgetting data by deviating the learnable generative process from the ground-truth denoising procedure. To solve the resulting problem, we adopt a first-order method, having superior practical performance while being vigilant about the diffusion process. Empirically, we demonstrate that our algorithm can preserve the model utility, effectiveness, and efficiency while removing across the widely-used diffusion models and in both conditional and unconditional image generation scenarios.

Related papers

• Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways [13.08168394252538]
  Erase inpainting aims to precisely remove target objects within masked regions while preserving the overall consistency of the surrounding content. We propose a novel Erase Diffusion, termed EraDiff, aimed at unleashing the potential power of standard diffusion in the context of object removal. Our proposed EraDiff achieves state-of-the-art performance on the OpenImages V5 dataset and demonstrates significant superiority in real-world scenarios.
  arXiv  Detail & Related papers  (2025-03-10T08:06:51Z)
• Mixed geometry information regularization for image multiplicative denoising [12.43943610396014]
  This paper focuses on solving the multiplicative gamma denoising problem via a variation model. To overcome these issues, in this paper we propose a mixed information model, incorporating area geometry term and curvature term as prior knowledge.
  arXiv  Detail & Related papers  (2024-12-21T02:24:42Z)
• Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models [93.76814568163353]
  We propose a novel bilevel optimization framework for pruned diffusion models. This framework consolidates the fine-tuning and unlearning processes into a unified phase. It is compatible with various pruning and concept unlearning methods.
  arXiv  Detail & Related papers  (2024-12-19T19:13:18Z)
• Diffusion Models as Optimizers for Efficient Planning in Offline RL [47.0835433289033]
  Diffusion models have shown strong competitiveness in offline reinforcement learning tasks. We propose a faster autoregressive model to handle the generation of feasible trajectories. This allows us to achieve more efficient planning without sacrificing capability.
  arXiv  Detail & Related papers  (2024-07-23T03:00:01Z)
• Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems [26.134240531687453]
  The ProjDiff algorithm harnesses the prior information and the denoising capability of a pre-trained diffusion model within the optimization framework. Experiments on the image restoration tasks and source separation and partial generation tasks demonstrate that ProjDiff exhibits superior performance across various linear and nonlinear inverse problems.
  arXiv  Detail & Related papers  (2024-06-11T05:35:18Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• The Missing U for Efficient Diffusion Models [3.712196074875643]
  Diffusion Probabilistic Models yield record-breaking performance in tasks such as image synthesis, video generation, and molecule design. Despite their capabilities, their efficiency, especially in the reverse process, remains a challenge due to slow convergence rates and high computational costs. We introduce an approach that leverages continuous dynamical systems to design a novel denoising network for diffusion models.
  arXiv  Detail & Related papers  (2023-10-31T00:12:14Z)
• Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation [25.55296442023984]
  We propose a method, Unlearnable Diffusion Perturbation, to safeguard images from unauthorized exploitation. This achievement holds significant importance in real-world scenarios, as it contributes to the protection of privacy and copyright against AI-generated content.
  arXiv  Detail & Related papers  (2023-06-02T20:19:19Z)
• dugMatting: Decomposed-Uncertainty-Guided Matting [83.71273621169404]
  We propose a decomposed-uncertainty-guided matting algorithm, which explores the explicitly decomposed uncertainties to efficiently and effectively improve the results. The proposed matting framework relieves the requirement for users to determine the interaction areas by using simple and efficient labeling.
  arXiv  Detail & Related papers  (2023-06-02T11:19:50Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Differentiable Causal Discovery from Interventional Data [141.41931444927184]
  We propose a theoretically-grounded method based on neural networks that can leverage interventional data. We show that our approach compares favorably to the state of the art in a variety of settings.
  arXiv  Detail & Related papers  (2020-07-03T15:19:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.