A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate

• URL: http://arxiv.org/abs/2608.02406v1
• Date: Mon, 03 Aug 2026 15:49:08 GMT
• Title: A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate
• Abstract summary: We introduce a unified framework for the error analysis of generative models based on the entropy production rate of the forward-reverse diffusion process pair.<n>By analyzing the entropy production at the level of marginal distributions, rather than in path space, our framework yields a sharp convergence rate of $mathcalO(h2)$ for the Euler-Maruyama sampler.
• Score: 4.625787677472786
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a unified framework for the error analysis of generative models based on the entropy production rate of the forward-reverse diffusion process pair. For a pair of continuity equation flows, the rate admits a closed velocity form identity whose time integral decomposes the terminal Kullback--Leibler (KL) divergence into the sum of an initialization error, a score approximation error, and a time-discretization error. By analyzing the entropy production at the level of marginal distributions, rather than in path space, our framework yields a sharp convergence rate of $\mathcal{O}(h^2)$ for the Euler-Maruyama sampler, where $h$ is the step size. This improves upon the $\mathcal{O}(h)$ rates typically obtained from Girsanov's path-space analyses. Furthermore, our framework unifies the analysis of score-based SDEs, probability-flow ODEs, and stochastic interpolants by varying diffusion coefficients within a single inequality, revealing the trade-off between deterministic and stochastic sampling. Numerical experiments confirm the predicted scaling with step size and terminal time.

Related papers

• A Quantitative Approximation Framework for Flow Distillation in Diffusion Models [9.704954388992892]
  Local approximation errors can be strongly amplified in low-noise multimodal regimes.<n>We show that deep residual compositions efficiently approximate the long-horizon transport.<n>Experiments support the prediction and reduce end-to-end relative MSE by up to 51.9% with 8 segments compared with uniform grids.
  arXiv  Detail & Related papers  (2026-06-02T16:00:43Z)
• From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models [58.02657021435213]
  We propose a novel Gibbs-based corrector for discrete diffusion models, termed Gibbs-Accelerated Discrete Diffusion (GADD)<n>We show GADD achieves an overall sampling complexity of $mathcalO(mathrmpolylog (varepsilon-1)$, yielding the first such rate for diffusion-based samplers for uniform-rate discrete diffusion models.
  arXiv  Detail & Related papers  (2026-05-26T17:52:28Z)
• Variational Optimality of Föllmer Processes in Generative Diffusions [20.583125441867434]
  We analyze generative diffusions that transport a point mass to a prescribed target distribution over a finite time horizon.<n>We show that the diffusion coefficient can be tuned emphaposteriori without changing the time-marginal distributions.
  arXiv  Detail & Related papers  (2026-02-11T16:15:19Z)
• An Elementary Approach to Scheduling in Generative Diffusion Models [55.171367482496755]
  An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed.<n> Experiments across different datasets and pretrained models demonstrate that the time discretization strategy selected by our approach consistently outperforms baseline and search-based strategies.
  arXiv  Detail & Related papers  (2026-01-20T05:06:26Z)
• Convergence, design and training of continuous-time dropout as a random batch method [0.0]
  We study dropout regularization in continuous-time models through the lens of random-batch methods.<n>We construct an unbiased, well-posed estimator that mimics dropout by sampling neuron batches over time intervals of length $h$.<n>We then specialize in a single-layer neural ODE and validate the theory on classification and flow matching.
  arXiv  Detail & Related papers  (2025-10-15T04:19:01Z)
• The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis [0.0]
  We study the effect of divergenceity on the generation process through bounds on the Kullback-Leibler (KL)<n>Our main results apply to linear forward SDEs with additive noise and Lipschitz-continuous score functions.
  arXiv  Detail & Related papers  (2025-06-13T01:01:07Z)
• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework.<n>We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points.<n>Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is hard to beat [49.1574468325115]
  This paper presents explicit non-asymptotic bounds on the forward diffusion error in total variation (TV)<n>We parametrise multi-modal data distributions in terms of the distance $R$ to their furthest modes and consider forward diffusions with additive and multiplicative noise.
  arXiv  Detail & Related papers  (2024-08-25T10:28:31Z)
• Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals [3.4240632942024685]
  We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score based MCMC method that is deterministic, resulting in a deterministic evolution for particles.
  arXiv  Detail & Related papers  (2023-08-28T23:51:33Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.