Understanding diffusion models requires rethinking (again) generalization

• URL: http://arxiv.org/abs/2605.06077v1
• Date: Thu, 07 May 2026 12:00:10 GMT
• Title: Understanding diffusion models requires rethinking (again) generalization
• Abstract summary: In diffusion models, unlike in supervised learning, memorization of training data and generalization to novel samples are incompatible.<n>We argue that the field should pivot from explaining why the diffusion models do not memorize to investigating what the model actually learns during pre-memorization phase.
• Score: 12.724875526837925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This position paper argues that understanding generalization in diffusion models requires fundamentally new theoretical frameworks that go beyond both classical statistical learning theory and the benign overfitting paradigm developed for supervised learning. In diffusion models, unlike in supervised learning, memorization of training data and generalization to novel samples are incompatible: a model that has fully memorized its training set generates copies rather than novel data. Several theoretical explanations for why practical diffusion models nevertheless generalize have been proposed, based on capacity limitations, implicit regularization from optimization, or architectural inductive biases, but their interactions remain unclear. We argue that the field should pivot from explaining why the diffusion models do not memorize to investigating what the model actually learns during pre-memorization phase. To highlight our stance, we conduct empirical study of diffusion models trained on CIFAR-10, and we distill the findings into concrete open questions that we believe are key to improve understanding of generalization in diffusion models.

Related papers

• On the Edge of Memorization in Diffusion Models [25.927892368310868]
  We introduce a scientific and mathematical "laboratory" for investigating memorization and generalization in practical diffusion models.<n>Our work provides an analytically tractable and practically meaningful setting for future theoretical and empirical investigations.
  arXiv  Detail & Related papers  (2025-08-25T05:56:05Z)
• Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Access [47.96419637803502]
  We present a principled theoretical framework analyzing diffusion models, providing a state-of-the-art sample complexity bound to $widetildemathcalO(epsilon-4)$.<n>Our structured decomposition of the score estimation error into statistical and optimization components offers critical insights into how diffusion models can be trained efficiently.
  arXiv  Detail & Related papers  (2025-05-23T20:02:15Z)
• Deeper Diffusion Models Amplify Bias [54.8794775172033]
  This paper focuses on exploring the notion of bias-variance tradeoff in diffusion models.<n>It establishes that at one extreme, the diffusion models may amplify the inherent bias in the training data, and on the other, they may compromise the presumed privacy of the training samples.
  arXiv  Detail & Related papers  (2025-05-23T07:08:09Z)
• Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models [56.032091696552094]
  Generalization in natural data domains is progressively achieved during training before the onset of memorization.<n>Generalization vs. memorization is then best understood as a competition between time scales.<n>We show that this phenomenology is recovered in diffusion models learning a simple probabilistic context-free grammar with random rules.
  arXiv  Detail & Related papers  (2025-05-22T17:40:08Z)
• Consistent World Models via Foresight Diffusion [56.45012929930605]
  We argue that a key bottleneck in learning consistent diffusion-based world models lies in the suboptimal predictive ability.<n>We propose Foresight Diffusion (ForeDiff), a diffusion-based world modeling framework that enhances consistency by decoupling condition understanding from target denoising.
  arXiv  Detail & Related papers  (2025-05-22T10:01:59Z)
• An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization [59.63880337156392]
  Diffusion models have achieved tremendous success in computer vision, audio, reinforcement learning, and computational biology. Despite the significant empirical success, theory of diffusion models is very limited. This paper provides a well-rounded theoretical exposure for stimulating forward-looking theories and methods of diffusion models.
  arXiv  Detail & Related papers  (2024-04-11T14:07:25Z)
• Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory [87.00653989457834]
  Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges the gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:08:24Z)
• Demystifying Variational Diffusion Models [19.977841588918373]
  Most existing work on diffusion models focuses on either applications or theoretical contributions.<n>We revisit predecessors to diffusion models like hierarchical latent variable models and synthesize a holistic perspective.<n>The resulting narrative is easier to follow as it imposes fewer prerequisites on the average reader.
  arXiv  Detail & Related papers  (2024-01-11T22:37:37Z)
• On the Generalization Properties of Diffusion Models [31.067038651873126]
  This work embarks on a comprehensive theoretical exploration of the generalization attributes of diffusion models.<n>We establish theoretical estimates of the generalization gap that evolves in tandem with the training dynamics of score-based diffusion models.<n>We extend our quantitative analysis to a data-dependent scenario, wherein target distributions are portrayed as a succession of densities.
  arXiv  Detail & Related papers  (2023-11-03T09:20:20Z)
• The Emergence of Reproducibility and Generalizability in Diffusion Models [10.188731323681575]
  Given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs. We show that diffusion models are learning distinct distributions affected by the training data size. This valuable property generalizes to many variants of diffusion models, including those for conditional use, solving inverse problems, and model fine-tuning.
  arXiv  Detail & Related papers  (2023-10-08T19:02:46Z)
• On the Generalization of Diffusion Model [42.447639515467934]
  We define the generalization of the generative model, which is measured by the mutual information between the generated data and the training set. We show that for the empirical optimal diffusion model, the data generated by a deterministic sampler are all highly related to the training set, thus poor generalization. We propose another training objective whose empirical optimal solution has no potential generalization problem.
  arXiv  Detail & Related papers  (2023-05-24T04:27:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.