A Free Probabilistic Framework for Denoising Diffusion Models: Entropy, Transport, and Reverse Processes

• URL: http://arxiv.org/abs/2510.22778v2
• Date: Sun, 02 Nov 2025 19:17:12 GMT
• Title: A Free Probabilistic Framework for Denoising Diffusion Models: Entropy, Transport, and Reverse Processes
• Authors: Swagatam Das,
• Abstract summary: This paper builds on Voiculescu's theory of free entropy and free Fisher information.<n>We formulate diffusion and quantify reverse processes governed by operator-valued dynamics.<n>The resulting dynamics admit a gradient-flow structure in the noncommutative Wasserstein space.
• Score: 22.56299060022639
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper develops a rigorous probabilistic framework that extends denoising diffusion models to the setting of noncommutative random variables. Building on Voiculescu's theory of free entropy and free Fisher information, we formulate diffusion and reverse processes governed by operator-valued stochastic dynamics whose spectral measures evolve by additive convolution. Using tools from free stochastic analysis -- including a Malliavin calculus and a Clark--Ocone representation -- we derive the reverse-time stochastic differential equation driven by the conjugate variable, the analogue of the classical score function. The resulting dynamics admit a gradient-flow structure in the noncommutative Wasserstein space, establishing an information-geometric link between entropy production, transport, and deconvolution. We further construct a variational scheme analogous to the Jordan--Kinderlehrer--Otto (JKO) formulation and prove convergence toward the semicircular equilibrium. The framework provides functional inequalities (free logarithmic Sobolev, Talagrand, and HWI) that quantify entropy dissipation and Wasserstein contraction. These results unify diffusion-based generative modeling with the geometry of operator-valued information, offering a mathematical foundation for generative learning on structured and high-dimensional data.

Related papers

• Kernel Methods for Stochastic Dynamical Systems with Application to Koopman Eigenfunctions: Feynman-Kac Representations and RKHS Approximation [3.2116198597240846]
  We extend the unified kernel framework for transport equations and Koopman eigenfunctions to differential equations (SDEs)<n>In the deterministic setting, three grounded approximations-Lions-type variational principles were shown to yield identical reproducing kernels.<n>For systems, the Koopman generator includes a second-order diffusion term, transforming the first-order hyperbolic transport equation into a second-order elliptic-parabolic PDE.
  arXiv  Detail & Related papers  (2026-03-01T12:44:31Z)
• Weighted Stochastic Differential Equation to Implement Wasserstein-Fisher-Rao Gradient Flow [0.0]
  A promising line of work leverages tools from information geometry to augment diffusion-based samplers with mass reweighting mechanisms.<n>Our work provides a preliminary but rigorous investigation of WFR-based sampling dynamics, and aims to clarify their theoretical structure as a foundation for future developments.
  arXiv  Detail & Related papers  (2025-12-19T18:31:27Z)
• Preconditioned Regularized Wasserstein Proximal Sampling [2.7957842724446174]
  We consider sampling from a noise-free distribution by evolving finitely many particles.<n>For potentials, we provide a non-asymotic convergence analysis and explicitly the bias, which is dependent on regularization.
  arXiv  Detail & Related papers  (2025-09-01T18:04:31Z)
• The Information Dynamics of Generative Diffusion [12.52425103385255]
  Generative diffusion models have emerged as a powerful class of models in machine learning.<n>This paper provides an integrated perspective on generative diffusion by connecting their dynamic, information-theoretic, and thermodynamic properties.
  arXiv  Detail & Related papers  (2025-08-27T13:53:56Z)
• Loss-Complexity Landscape and Model Structure Functions [53.92822954974537]
  We develop a framework for dualizing the Kolmogorov structure function $h_x(alpha)$.<n>We establish a mathematical analogy between information-theoretic constructs and statistical mechanics.<n>We explicitly prove the Legendre-Fenchel duality between the structure function and free energy.
  arXiv  Detail & Related papers  (2025-07-17T21:31:45Z)
• Overcoming Dimensional Factorization Limits in Discrete Diffusion Models through Quantum Joint Distribution Learning [79.65014491424151]
  We propose a quantum Discrete Denoising Diffusion Probabilistic Model (QD3PM)<n>It enables joint probability learning through diffusion and denoising in exponentially large Hilbert spaces.<n>This paper establishes a new theoretical paradigm in generative models by leveraging the quantum advantage in joint distribution learning.
  arXiv  Detail & Related papers  (2025-05-08T11:48:21Z)
• Hessian-Informed Flow Matching [4.542719108171107]
  Hessian-Informed Flow Matching is a novel approach that integrates the Hessian of an energy function into conditional flows. This integration allows HI-FM to account for local curvature and anisotropic covariance structures. Empirical evaluations on the MNIST and Lennard-Jones particles datasets demonstrate that HI-FM improves the likelihood of test samples.
  arXiv  Detail & Related papers  (2024-10-15T09:34:52Z)
• Discrete generative diffusion models without stochastic differential equations: a tensor network approach [1.5839621757142595]
  Diffusion models (DMs) are a class of generative machine learning methods. We show how to use networks (TNs) to efficiently define and sample such discrete models''
  arXiv  Detail & Related papers  (2024-07-15T18:00:11Z)
• Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models [59.331993845831946]
  Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
  arXiv  Detail & Related papers  (2024-03-03T23:15:48Z)
• Logistic-beta processes for dependent random probabilities with beta marginals [58.91121576998588]
  We propose a novel process called the logistic-beta process, whose logistic transformation yields a process with common beta marginals.<n>We show how the proposed logistic-beta process can be used to design computationally tractable dependent nonparametric models.
  arXiv  Detail & Related papers  (2024-02-10T21:41:32Z)
• Stochastic Schr\"odinger equation approach to real-time dynamics of Anderson-Holstein impurities: an open quantum system perspective [3.105656247358225]
  We develop a framework to simulate dynamics of Anderson-Holstein impurities coupled to a continuous fermionic bath. We show that such an SSE treatment provides a middle ground between numerically expansive microscopic simulations and master equations.
  arXiv  Detail & Related papers  (2023-09-16T06:03:54Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.