Memorization Control in Diffusion Models from Denoising-centric Perspective

• URL: http://arxiv.org/abs/2601.21348v1
• Date: Thu, 29 Jan 2026 07:16:54 GMT
• Title: Memorization Control in Diffusion Models from Denoising-centric Perspective
• Authors: Thuy Phuong Vu, Mai Viet Hoang Do, Minhhuy Le, Dinh-Cuong Hoang, Phan Xuan Tan,
• Abstract summary: Controlling memorization in diffusion models is critical for applications that require generated data to closely match the training distribution.<n>We show that uniform timestep sampling leads to unequal learning contributions across denoising steps due to differences in signal to noise ratio.<n>We propose a timestep sampling strategy that explicitly controls where learning occurs along the denoising trajectory.
• Score: 0.6741942263052466
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Controlling memorization in diffusion models is critical for applications that require generated data to closely match the training distribution. Existing approaches mainly focus on data centric or model centric modifications, treating the diffusion model as an isolated predictor. In this paper, we study memorization in diffusion models from a denoising centric perspective. We show that uniform timestep sampling leads to unequal learning contributions across denoising steps due to differences in signal to noise ratio, which biases training toward memorization. To address this, we propose a timestep sampling strategy that explicitly controls where learning occurs along the denoising trajectory. By adjusting the width of the confidence interval, our method provides direct control over the memorization generalization trade off. Experiments on image and 1D signal generation tasks demonstrate that shifting learning emphasis toward later denoising steps consistently reduces memorization and improves distributional alignment with training data, validating the generality and effectiveness of our approach.

Related papers

• Detecting and Mitigating Memorization in Diffusion Models through Anisotropy of the Log-Probability [9.133729396364952]
  Diffusion-based image generative models produce high-fidelity images through iterative denoising but remain vulnerable to memorization.<n>Recent memorization detection methods are primarily based on the norm of score difference as indicators of memorization.<n>We develop a memorization detection metric by integrating isotropic norm and anisotropic alignment.
  arXiv  Detail & Related papers  (2026-01-28T14:29:42Z)
• The Principles of Diffusion Models [81.12042238390075]
  Diffusion modeling starts by defining a forward process that gradually corrupts data into noise.<n>The goal is to learn a reverse process that transforms noise back into data while recovering the same intermediates.<n>The score-based view, rooted in energy-based modeling, learns the gradient of the evolving data distribution.<n>The flow-based view, related to normalizing flows, treats generation as following a smooth path that moves samples from noise to data.
  arXiv  Detail & Related papers  (2025-10-24T02:29:02Z)
• How Diffusion Models Memorize [26.711679643772623]
  diffusion models can memorize training data, raising serious privacy and copyright concerns.<n>We show memorization is driven by the overestimation of training samples during early denoising.
  arXiv  Detail & Related papers  (2025-09-30T03:03:27Z)
• Guiding Noisy Label Conditional Diffusion Models with Score-based Discriminator Correction [1.8434042562191815]
  This paper introduces Score-based Discriminator Correction (SBDC), a guidance technique for aligning noisy pre-trained conditional diffusion models.<n>Our method is computationally efficient, only marginally increases inference time, and does not require retraining diffusion models.
  arXiv  Detail & Related papers  (2025-08-27T05:29:07Z)
• NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation [66.36912000442608]
  NoisyRollout is a simple yet effective data augmentation method.<n>It mixes training trajectories from both clean and moderately distorted images.<n>It achieves state-of-the-art performance among open-source RL-tuned models.
  arXiv  Detail & Related papers  (2025-04-17T16:10:13Z)
• ADT: Tuning Diffusion Models with Adversarial Supervision [16.974169058917443]
  Diffusion models have achieved outstanding image generation by reversing a forward noising process to approximate true data distributions.<n>We propose Adrial Diffusion Tuning (ADT) to stimulate the inference process during optimization and align the final outputs with training data.<n>ADT features a siamese-network discriminator with a fixed pre-trained backbone and lightweight trainable parameters.
  arXiv  Detail & Related papers  (2025-04-15T17:37:50Z)
• Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data [74.2507346810066]
  Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. We present the first framework for training diffusion models that provably sample from the uncorrupted distribution given only noisy training data.
  arXiv  Detail & Related papers  (2024-03-20T14:22:12Z)
• Erasing Undesirable Influence in Diffusion Models [51.225365010401006]
  Diffusion models are highly effective at generating high-quality images but pose risks, such as the unintentional generation of NSFW (not safe for work) content. In this work, we introduce EraseDiff, an algorithm designed to preserve the utility of the diffusion model on retained data while removing the unwanted information associated with the data to be forgotten.
  arXiv  Detail & Related papers  (2024-01-11T09:30:36Z)
• Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Models [52.1809084559048]
  We propose a novel two-stage divide-and-conquer training strategy termed TDC Training.<n>It groups timesteps based on task similarity and difficulty, assigning highly customized denoising models to each group, thereby enhancing the performance of diffusion models.<n>While two-stage training avoids the need to train each model separately, the total training cost is even lower than training a single unified denoising model.
  arXiv  Detail & Related papers  (2023-12-20T03:32:58Z)
• One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls [77.42510898755037]
  One More Step (OMS) is a compact network that incorporates an additional simple yet effective step during inference. OMS elevates image fidelity and harmonizes the dichotomy between training and inference, while preserving original model parameters. Once trained, various pre-trained diffusion models with the same latent domain can share the same OMS module.
  arXiv  Detail & Related papers  (2023-11-27T12:02:42Z)
• Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Models [63.1637853118899]
  We propose the first unsupervised and learning-based method to identify interpretable directions in h-space of pre-trained diffusion models. We employ a shift control module that works on h-space of pre-trained diffusion models to manipulate a sample into a shifted version of itself. By jointly optimizing them, the model will spontaneously discover disentangled and interpretable directions.
  arXiv  Detail & Related papers  (2023-10-15T18:44:30Z)
• Observation-Guided Diffusion Probabilistic Models [41.749374023639156]
  We propose a novel diffusion-based image generation method called the observation-guided diffusion probabilistic model (OGDM) Our approach reestablishes the training objective by integrating the guidance of the observation process with the Markov chain. We demonstrate the effectiveness of our training algorithm using diverse inference techniques on strong diffusion model baselines.
  arXiv  Detail & Related papers  (2023-10-06T06:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.