Detecting and Mitigating Memorization in Diffusion Models through Anisotropy of the Log-Probability

• URL: http://arxiv.org/abs/2601.20642v1
• Date: Wed, 28 Jan 2026 14:29:42 GMT
• Title: Detecting and Mitigating Memorization in Diffusion Models through Anisotropy of the Log-Probability
• Authors: Rohan Asthana, Vasileios Belagiannis,
• Abstract summary: Diffusion-based image generative models produce high-fidelity images through iterative denoising but remain vulnerable to memorization.<n>Recent memorization detection methods are primarily based on the norm of score difference as indicators of memorization.<n>We develop a memorization detection metric by integrating isotropic norm and anisotropic alignment.
• Score: 9.133729396364952
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion-based image generative models produce high-fidelity images through iterative denoising but remain vulnerable to memorization, where they unintentionally reproduce exact copies or parts of training images. Recent memorization detection methods are primarily based on the norm of score difference as indicators of memorization. We prove that such norm-based metrics are mainly effective under the assumption of isotropic log-probability distributions, which generally holds at high or medium noise levels. In contrast, analyzing the anisotropic regime reveals that memorized samples exhibit strong angular alignment between the guidance vector and unconditional scores in the low-noise setting. Through these insights, we develop a memorization detection metric by integrating isotropic norm and anisotropic alignment. Our detection metric can be computed directly on pure noise inputs via two conditional and unconditional forward passes, eliminating the need for costly denoising steps. Detection experiments on Stable Diffusion v1.4 and v2 show that our metric outperforms existing denoising-free detection methods while being at least approximately 5x faster than the previous best approach. Finally, we demonstrate the effectiveness of our approach by utilizing a mitigation strategy that adapts memorized prompts based on our developed metric.

Related papers

• Memorization Control in Diffusion Models from Denoising-centric Perspective [0.6741942263052466]
  Controlling memorization in diffusion models is critical for applications that require generated data to closely match the training distribution.<n>We show that uniform timestep sampling leads to unequal learning contributions across denoising steps due to differences in signal to noise ratio.<n>We propose a timestep sampling strategy that explicitly controls where learning occurs along the denoising trajectory.
  arXiv  Detail & Related papers  (2026-01-29T07:16:54Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Catching Contamination Before Generation: Spectral Kill Switches for Agents [0.0]
  We introduce a diagnostic that uses only the forward pass to emit a binary accept or reject signal during agent execution.<n>The method analyzes token graphs induced by attention and computes two spectral statistics in early layers.<n>We show that a single threshold on the high frequency energy ratio is optimal in the Bayes sense for detecting context inconsistency.
  arXiv  Detail & Related papers  (2025-11-08T02:24:05Z)
• TAG:Tangential Amplifying Guidance for Hallucination-Resistant Diffusion Sampling [53.61290359948953]
  Tangential Amplifying Guidance (TAG) operates solely on trajectory signals without modifying the underlying diffusion model.<n>We formalize this guidance process by leveraging a first-order Taylor expansion.<n> TAG is a plug-and-play, architecture-agnostic module that improves diffusion sampling fidelity with minimal computational addition.
  arXiv  Detail & Related papers  (2025-10-06T06:53:29Z)
• Noise Conditional Variational Score Distillation [60.38982038894823]
  Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
  arXiv  Detail & Related papers  (2025-06-11T06:01:39Z)
• Reconstruction-Free Anomaly Detection with Diffusion Models [30.099399014193573]
  We propose a novel inversion-based anomaly detection (AD) approach - detection via noising in latent space.<n>In approximating the original probability flow ODE, we only enforce very few inversion steps to noise the clean image.<n>As the added noise is adaptively derived with the learned diffusion model, the original features for the clean testing image can still be leveraged to yield high detection accuracy.
  arXiv  Detail & Related papers  (2025-04-08T04:23:43Z)
• Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization [11.088273093231324]
  We show that when trained by gradient descent with a large enough learning rate, neural networks cannot converge to a local minimum with small excess risk.<n>Experiments validate the crucial role of the learning rate in preventing memorization, even beyond the one-dimensional setting.
  arXiv  Detail & Related papers  (2025-02-05T18:29:35Z)
• Robust Representation Consistency Model via Contrastive Denoising [83.47584074390842]
  randomized smoothing provides theoretical guarantees for certifying robustness against adversarial perturbations.<n> diffusion models have been successfully employed for randomized smoothing to purify noise-perturbed samples.<n>We reformulate the generative modeling task along the diffusion trajectories in pixel space as a discriminative task in the latent space.
  arXiv  Detail & Related papers  (2025-01-22T18:52:06Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [80.20339155618612]
  DiffusionAD is a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network.<n>A rapid one-step denoising paradigm achieves hundreds of times acceleration while preserving comparable reconstruction quality.<n>Considering the diversity in the manifestation of anomalies, we propose a norm-guided paradigm to integrate the benefits of multiple noise scales.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Iterative Correction of Sensor Degradation and a Bayesian Multi-Sensor Data Fusion Method [0.0]
  We present a novel method for inferring ground-truth signal from degraded signals. The algorithm learns a multiplicative degradation effect by performing iterative corrections of two signals. We include theoretical analysis and prove convergence to the ground-truth signal for the noiseless measurement model.
  arXiv  Detail & Related papers  (2020-09-07T13:24:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.