Provable Separations between Memorization and Generalization in Diffusion Models

• URL: http://arxiv.org/abs/2511.03202v2
• Date: Fri, 07 Nov 2025 00:14:54 GMT
• Title: Provable Separations between Memorization and Generalization in Diffusion Models
• Authors: Zeqi Ye, Qijie Zhu, Molei Tao, Minshuo Chen,
• Abstract summary: We develop a dual-separation result via statistical estimation and network approximation.<n>We show that the ground-truth score function does not minimize the empirical denoising loss, creating a separation that drives memorization.
• Score: 31.32606216124959
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models have achieved remarkable success across diverse domains, but they remain vulnerable to memorization -- reproducing training data rather than generating novel outputs. This not only limits their creative potential but also raises concerns about privacy and safety. While empirical studies have explored mitigation strategies, theoretical understanding of memorization remains limited. We address this gap through developing a dual-separation result via two complementary perspectives: statistical estimation and network approximation. From the estimation side, we show that the ground-truth score function does not minimize the empirical denoising loss, creating a separation that drives memorization. From the approximation side, we prove that implementing the empirical score function requires network size to scale with sample size, spelling a separation compared to the more compact network representation of the ground-truth score function. Guided by these insights, we develop a pruning-based method that reduces memorization while maintaining generation quality in diffusion transformers.

Related papers

• Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information [55.75102049412629]
  We show that effective unlearnable examples always decrease mutual information between clean features and poisoned features.<n>We propose a novel unlearnable method called Mutual Information Unlearnable Examples (MI-UE)<n>Our approach significantly outperforms the previous methods, even under defense mechanisms.
  arXiv  Detail & Related papers  (2026-03-04T04:53:29Z)
• Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects [17.220195638215507]
  Diffusion language models (DLMs) have emerged as a competitive alternative to autoregressive language models (ARMs)<n>DLMs exhibit substantially lower memorization-based leakage of personally identifiable information (PII) compared to ARMs.
  arXiv  Detail & Related papers  (2026-03-02T19:03:32Z)
• Latent Diffusion Inversion Requires Understanding the Latent Space [3.742113529511043]
  encoder/decoder pair and corresponding latent codes have largely been ignored by inversion techniques applied to latent space generative models.<n>We show that the diffusion model exhibits non-uniform memorization across latent codes, tending to overfit samples located in high-distortion regions.<n>We introduce a principled method to rank latent dimensions by their per-dimensional contribution to the decoder pullback metric, identifying those most responsible for memorization.
  arXiv  Detail & Related papers  (2025-11-25T18:21:33Z)
• Finding Dori: Memorization in Text-to-Image Diffusion Models Is Not Local [55.33447817350623]
  Recent mitigation efforts have focused on identifying and pruning weights responsible for triggering verbatim training data replication.<n>We challenge this assumption and demonstrate that, even after such pruning, small perturbations to the text embeddings of previously mitigated prompts can re-trigger data replication.<n>Our findings provide new insights into the nature of memorization in text-to-image DMs and inform the development of more reliable mitigations against DM memorization.
  arXiv  Detail & Related papers  (2025-07-22T15:02:38Z)
• Redistribute Ensemble Training for Mitigating Memorization in Diffusion Models [31.92526915009259]
  Diffusion models are known for their tremendous ability to generate high-quality samples.<n>Recent methods for memory mitigation have primarily addressed the issue within the context of the text modality.<n>We propose a novel method for diffusion models from the perspective of visual modality, which is more generic and fundamental for mitigating memorization.
  arXiv  Detail & Related papers  (2025-02-13T15:56:44Z)
• Detecting, Explaining, and Mitigating Memorization in Diffusion Models [49.438362005962375]
  We introduce a straightforward yet effective method for detecting memorized prompts by inspecting the magnitude of text-conditional predictions. Our proposed method seamlessly integrates without disrupting sampling algorithms, and delivers high accuracy even at the first generation step. Building on our detection strategy, we unveil an explainable approach that shows the contribution of individual words or tokens to memorization.
  arXiv  Detail & Related papers  (2024-07-31T16:13:29Z)
• Iterative Ensemble Training with Anti-Gradient Control for Mitigating Memorization in Diffusion Models [20.550324116099357]
  Diffusion models are known for their tremendous ability to generate novel and high-quality samples.<n>Recent approaches for memory mitigation either only focused on the text modality problem in cross-modal generation tasks or utilized data augmentation strategies.<n>We propose a novel training framework for diffusion models from the perspective of visual modality, which is more generic and fundamental for mitigating memorization.
  arXiv  Detail & Related papers  (2024-07-22T02:19:30Z)
• An Inversion-based Measure of Memorization for Diffusion Models [37.9715620828388]
  diffusion models are susceptible to training data memorization, raising concerns regarding copyright infringement and privacy invasion.<n>We introduce InvMM, an inversion-based measure of memorization, which is based on inverting a sensitive latent noise distribution accounting for the replication of an image.<n>InvMM is commensurable between samples, reveals the true extent of memorization from an adversarial standpoint and implies how memorization differs from membership.
  arXiv  Detail & Related papers  (2024-05-09T15:32:00Z)
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution [67.9215891673174]
  We propose score entropy as a novel loss that naturally extends score matching to discrete spaces. We test our Score Entropy Discrete Diffusion models on standard language modeling tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:59:12Z)
• Cross-Domain Similarity Learning for Face Recognition in Unseen Domains [90.35908506994365]
  We introduce a novel cross-domain metric learning loss, which we dub Cross-Domain Triplet (CDT) loss, to improve face recognition in unseen domains. The CDT loss encourages learning semantically meaningful features by enforcing compact feature clusters of identities from one domain. Our method does not require careful hard-pair sample mining and filtering strategy during training.
  arXiv  Detail & Related papers  (2021-03-12T19:48:01Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)
• Towards Certified Robustness of Distance Metric Learning [53.96113074344632]
  We advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms. We show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness.
  arXiv  Detail & Related papers  (2020-06-10T16:51:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.