Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions

• URL: http://arxiv.org/abs/2605.30153v1
• Date: Thu, 28 May 2026 16:13:44 GMT
• Title: Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions
• Abstract summary: We study the sample complexity of diffusion models for learning distributions supported on a union of low-dimensional subspaces.<n>We show that diffusion models require at most $widetildeO(varepsilon-k vee 2)$ samples to achieve $varepsilon$ error in 1-Wasserstein distance, where $k$ is the intrinsic dimension.
• Score: 11.621528352410502
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Score-based diffusion models have demonstrated remarkable empirical success in learning high-dimensional distributions, particularly those exhibiting low-dimensional and multi-modal structures. However, theoretical understanding of their statistical efficiency remains limited. Existing theories typically rely on strong regularity assumptions, such as uniformly bounded densities or globally smooth score functions, which fail to capture such intrinsic structures. In this work, we study the sample complexity of diffusion models for learning distributions supported on a union of low-dimensional subspaces. Assuming that the data distribution within each subspace is subgaussian, we show that diffusion models require at most $\widetilde{O}(\varepsilon^{-k \vee 2})$ samples to achieve $\varepsilon$ error in 1-Wasserstein distance, where $k$ is the intrinsic dimension. This near-optimal convergence rate depends only on the intrinsic dimension and significantly improves upon prior theoretical guarantees that suffer from the curse of dimensionality. Notably, our analysis applies to a broad collection of distributions without imposing smoothness, bounded-density, or log-concavity assumptions. Overall, our results show that diffusion models can statistically adapt to intrinsic low-dimensional structure while naturally accommodating multi-modal data, offering a rigorous theoretical justification for their success in complex high-dimensional learning tasks.

Related papers

• Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures [23.118804869806898]
  This work establishes a universal score approximation theorem that works for any distribution supported on any compact set of upper Minkowski dimension $d$.<n>Using a novel discrete-mixture formulation, we prove that the score function can be approximated with a ReLU network whose complexity grows exponentially only with $d$.
  arXiv  Detail & Related papers  (2026-06-18T07:51:33Z)
• Reflected diffusion models adapt to low-dimensional data [1.4273866043218157]
  This paper provides a statistical analysis of reflected diffusion models on the hypercube $[0,1]D$ supported on $d$-dimensional linear subspaces.<n>By employing an easily implementable infinite series expansion of the transition densities, we develop analytic tools to bound the score function and its approximation by sparse ReLU networks.
  arXiv  Detail & Related papers  (2026-03-25T16:37:33Z)
• Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity [93.52679474816728]
  Diffusion models have become a leading framework in generative modeling.<n>This paper investigates how diffusion models learn structured data, focusing on two key aspects: statistical complexity and influence of data geometric properties.<n>We also highlight the interplay of manifold curvature with the structures in the score function.
  arXiv  Detail & Related papers  (2026-03-21T04:40:19Z)
• Discrete State Diffusion Models: A Sample Complexity Perspective [43.61958734990224]
  We present a principled theoretical framework for discrete-state diffusion, providing the first sample complexity bound of $widetildemathcalO(epsilon-2)$.<n>Our structured decomposition of the score estimation error into statistical, approximation, optimization, and clipping components offers critical insights into how discrete-state models can be trained efficiently.
  arXiv  Detail & Related papers  (2025-10-12T23:33:46Z)
• Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Access [47.96419637803502]
  We present a principled theoretical framework analyzing diffusion models, providing a state-of-the-art sample complexity bound to $widetildemathcalO(epsilon-4)$.<n>Our structured decomposition of the score estimation error into statistical and optimization components offers critical insights into how diffusion models can be trained efficiently.
  arXiv  Detail & Related papers  (2025-05-23T20:02:15Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models [6.76974373198208]
  We find that the dependency of the error incurred within each denoising step on the ambient dimension $d$ is in general unavoidable.<n>This represents the first theoretical demonstration that the DDPM sampler can adapt to unknown low-dimensional structures in the target distribution.
  arXiv  Detail & Related papers  (2024-05-23T17:59:10Z)
• Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory [87.00653989457834]
  Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges the gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:08:24Z)
• Diffusion Models are Minimax Optimal Distribution Estimators [49.47503258639454]
  We provide the first rigorous analysis on approximation and generalization abilities of diffusion modeling. We show that when the true density function belongs to the Besov space and the empirical score matching loss is properly minimized, the generated data distribution achieves the nearly minimax optimal estimation rates.
  arXiv  Detail & Related papers  (2023-03-03T11:31:55Z)
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data [68.62134204367668]
  This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. The generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
  arXiv  Detail & Related papers  (2023-02-14T17:02:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.