Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine

• URL: http://arxiv.org/abs/2605.20235v1
• Date: Sat, 16 May 2026 16:51:10 GMT
• Title: Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
• Abstract summary: Diffusion models generate high-dimensional data with remarkable quality.<n>How their training efficiently learns the score function remains theoretically unexplained.<n>We instantiate this principle as Score-induced Latent Diffusion (SiLD)
• Score: 60.669081685261965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models generate high-dimensional data with remarkable quality, yet how their training efficiently learns the score function, bypassing the curse of dimensionality when data is supported on low-dimensional manifolds, remains theoretically unexplained. We identify a collapse-and-refine mechanism driven by the geometry of the score function itself: at small noise scales, the diverging singularity of the score drives a rapid dimensional collapse of the induced denoising map onto the data manifold projection; at moderate noise scales, training refines the intrinsic density on the learned manifold. We instantiate this principle as Score-induced Latent Diffusion (SiLD), a two-stage framework in which both manifold learning and density estimation emerge from a single denoising score matching objective, replacing the heuristic KL regularization of VAE-based latent diffusion models. We prove that the resulting sample complexity depends on the intrinsic dimension rather than the ambient dimension. Experiments on Stacked MNIST, CelebA variants, and molecular generation benchmarks show that SiLD matches or outperforms VAE-based LDMs in generation quality and consistently improves reconstruction, validating our theoretical predictions.

Related papers

• MIND: Decoupling Model-Induced Label Noise via Latent Manifold Disentanglement [3.626013617212667]
  Model-induced label noise dominates noise in classical robust learning.<n>We propose Model-Induced Noise Decoupling (MIND), a theoretically grounded framework addressing this dilemma.<n>MIND significantly outperforms state-of-the-art methods on complex benchmarks.
  arXiv  Detail & Related papers  (2026-05-15T15:44:19Z)
• Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Data [7.047721358838963]
  We study the theoretical behavior of denoising score matching.<n>We derive exact expressions for the test, train, and score errors in the high-dimensional limit.
  arXiv  Detail & Related papers  (2026-03-24T08:58:12Z)
• Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity [93.52679474816728]
  Diffusion models have become a leading framework in generative modeling.<n>This paper investigates how diffusion models learn structured data, focusing on two key aspects: statistical complexity and influence of data geometric properties.<n>We also highlight the interplay of manifold curvature with the structures in the score function.
  arXiv  Detail & Related papers  (2026-03-21T04:40:19Z)
• Multimodal Atmospheric Super-Resolution With Deep Generative Models [1.9367648935513015]
  Score-based diffusion modeling is a generative machine learning algorithm that can be used to sample from complex distributions.<n>In this article, we apply such a concept to the super-resolution of a high-dimensional dynamical system, given the real-time availability of low-resolution and experimentally observed sparse sensor measurements.
  arXiv  Detail & Related papers  (2025-06-28T06:47:09Z)
• Overcoming Dimensional Factorization Limits in Discrete Diffusion Models through Quantum Joint Distribution Learning [79.65014491424151]
  We propose a quantum Discrete Denoising Diffusion Probabilistic Model (QD3PM)<n>It enables joint probability learning through diffusion and denoising in exponentially large Hilbert spaces.<n>This paper establishes a new theoretical paradigm in generative models by leveraging the quantum advantage in joint distribution learning.
  arXiv  Detail & Related papers  (2025-05-08T11:48:21Z)
• A solvable model of learning generative diffusion: theory and insights [44.979670733807694]
  We consider the problem of learning a flow or diffusion-based generative model parametrized by a two-layer auto-encoder.<n>We derive a tight characterization of low-dimensional projections of the distribution of samples generated by the learned model.
  arXiv  Detail & Related papers  (2025-01-07T16:56:40Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models [76.76860707897413]
  Current best practice advocates for a large T to ensure that the forward dynamics brings the diffusion sufficiently close to a known and simple noise distribution. We show how an auxiliary model can be used to bridge the gap between the ideal and the simulated forward dynamics, followed by a standard reverse diffusion process.
  arXiv  Detail & Related papers  (2022-06-10T15:09:46Z)
• Diagnosing and Fixing Manifold Overfitting in Deep Generative Models [11.82509693248749]
  Likelihood-based, or explicit, deep generative models use neural networks to construct flexible high-dimensional densities. We show that observed data lies on a low-dimensional manifold embedded in high-dimensional ambient space. We propose a class of two-step procedures consisting of a dimensionality reduction step followed by maximum-likelihood density estimation.
  arXiv  Detail & Related papers  (2022-04-14T18:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.