When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory

• URL: http://arxiv.org/abs/2605.07969v1
• Date: Fri, 08 May 2026 16:30:37 GMT
• Title: When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory
• Abstract summary: Diffusion models perform remarkably well on high-dimensional data such as images.<n>Existing convergence theory does not fully explain why such samplers remain efficient in high dimensions.<n>We show that diffusion sampling can remain efficient in high-dimensional spaces when the data distribution admits a compact latent representation.
• Score: 2.205829309604458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models perform remarkably well on high-dimensional data such as images, often using only a modest number of reverse-time steps. Despite this practical success, existing convergence theory does not fully explain why such samplers remain efficient in high dimensions. Many prior KL guarantees bound the discretization error in terms of the ambient dimension, while other improved results replace this dependence using intrinsic-dimensional or geometric structure assumptions. In this work, we develop an alternative information-theoretic perspective on diffusion sampler convergence. We prove that, for Gaussian mixture targets, the discretization error is controlled by the Shannon entropy of the latent mixture component rather than by the ambient dimension. Consequently, the leading step complexity scales linearly with latent entropy and depends only logarithmically on the second moment of the data. Our analysis also extends to discrete target distributions, where the relevant complexity is the entropy of the target rather than the dimension of the embedding space. These results suggest that diffusion sampling can remain efficient in high-dimensional spaces when the data distribution admits a compact latent representation, as is widely believed to be the case for natural images.

Related papers

• Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions [11.621528352410502]
  We study the sample complexity of diffusion models for learning distributions supported on a union of low-dimensional subspaces.<n>We show that diffusion models require at most $widetildeO(varepsilon-k vee 2)$ samples to achieve $varepsilon$ error in 1-Wasserstein distance, where $k$ is the intrinsic dimension.
  arXiv  Detail & Related papers  (2026-05-28T16:13:44Z)
• Geometry-Aware Discretization Error of Diffusion Models [24.297736470921915]
  Under a fixed inference budget, one must simulate a reverse-time ODE or SDE using only a limited number of denoising steps.<n>Existing non-asymptotic analyses provide convergence guarantees, but are typically too loose and too insensitive to diffusion parameters to guide practical design.<n>We derive first-order expansions of the Euler-Maruyama weak and Fréchet discretization errors.
  arXiv  Detail & Related papers  (2026-05-08T19:02:21Z)
• Polynomial Convergence of Riemannian Diffusion Models [46.72936436234762]
  Diffusion models are considered one of the state-of-the-art generative models in modern AI.<n>Most of the existing literature assumes that the underlying space is Euclidean.<n>In many practical applications, the data are constrained to lie on a submanifold of Euclidean space.
  arXiv  Detail & Related papers  (2026-01-05T19:14:09Z)
• Overcoming Dimensional Factorization Limits in Discrete Diffusion Models through Quantum Joint Distribution Learning [79.65014491424151]
  We propose a quantum Discrete Denoising Diffusion Probabilistic Model (QD3PM)<n>It enables joint probability learning through diffusion and denoising in exponentially large Hilbert spaces.<n>This paper establishes a new theoretical paradigm in generative models by leveraging the quantum advantage in joint distribution learning.
  arXiv  Detail & Related papers  (2025-05-08T11:48:21Z)
• Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces [5.716752583983991]
  When the data distribution consists of n points, empirical diffusion models tend to reproduce existing data points.<n>This work shows that the memorization issue can be solved simply by applying an inertia update at the end of the empirical diffusion simulation.<n>We demonstrate that the distribution of samples from this model approximates the true data distribution on a $C2$ manifold of dimension $d$, within a Wasserstein-1 distance of order $O(n-frac2d+4)$.
  arXiv  Detail & Related papers  (2025-05-05T09:40:41Z)
• Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures [18.828955620788566]
  Diffusion models are distinguished by their exceptional generative performance.<n>This paper investigates the effectiveness of diffusion models in sampling from complex high-dimensional distributions.
  arXiv  Detail & Related papers  (2025-04-07T17:59:07Z)
• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework.<n>We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points.<n>Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions [6.9408143976091745]
  Denoising Diffusion Probabilistic Models (DDPM) are powerful state-of-the-art methods used to generate synthetic data from high-dimensional data distributions.<n>We study DDPMs under the manifold hypothesis and prove that they achieve rates independent of the ambient dimension in terms of score learning.<n>In terms of sampling complexity, we obtain rates independent of the ambient dimension w.r.t. the Kullback-Leibler divergence, and $O(sqrtD)$ w.r.t. the Wasserstein distance.
  arXiv  Detail & Related papers  (2024-09-27T14:57:18Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Diffusion Models are Minimax Optimal Distribution Estimators [49.47503258639454]
  We provide the first rigorous analysis on approximation and generalization abilities of diffusion modeling. We show that when the true density function belongs to the Besov space and the empirical score matching loss is properly minimized, the generated data distribution achieves the nearly minimax optimal estimation rates.
  arXiv  Detail & Related papers  (2023-03-03T11:31:55Z)
• Infinite-Dimensional Diffusion Models [3.9748528039819977]
  We formulate diffusion-based generative models in infinite dimensions and apply them to the generative modelling of functions.<n>Using our theory, we also develop guidelines for the design of infinite-dimensional diffusion models.
  arXiv  Detail & Related papers  (2023-02-20T18:00:38Z)
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data [68.62134204367668]
  This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. The generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
  arXiv  Detail & Related papers  (2023-02-14T17:02:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.