Understanding Diffusion Models by Feynman's Path Integral

• URL: http://arxiv.org/abs/2403.11262v1
• Date: Sun, 17 Mar 2024 16:24:29 GMT
• Title: Understanding Diffusion Models by Feynman's Path Integral
• Authors: Yuji Hirono, Akinori Tanaka, Kenji Fukushima,
• Abstract summary: We introduce a novel formulation of diffusion models using Feynman's integral path. We find this formulation providing comprehensive descriptions of score-based generative models. We also demonstrate the derivation of backward differential equations and loss functions.
• Score: 2.4373900721120285
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman's path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions.The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck's constant in quantum physics. This analogy enables us to apply the Wentzel-Kramers-Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.

Related papers

• Stochastic quantization and diffusion models [0.0]
  This is a review of the possible connection between the quantization in physics and the diffusion models in machine learning. For machine-learning applications, the denoising diffusion model has been established as a successful technique. In this review, we focus on an SDE approach used in the score-based generative modeling.
  arXiv  Detail & Related papers  (2024-11-18T05:47:41Z)
• Stochastic Sampling from Deterministic Flow Models [8.849981177332594]
  We present a method to turn flow models into a family of differential equations (SDEs) that have the same marginal distributions. We empirically demonstrate advantages of our method on a toy Gaussian setup and on the large scale ImageNet generation task.
  arXiv  Detail & Related papers  (2024-10-03T05:18:28Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models [59.331993845831946]
  Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
  arXiv  Detail & Related papers  (2024-03-03T23:15:48Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Stochastic Interpolants: A Unifying Framework for Flows and Diffusions [16.95541777254722]
  A class of generative models that unifies flow-based and diffusion-based methods is introduced. These models extend the framework proposed in Albergo & VandenEijnden (2023), enabling the use of a broad class of continuous-time processes called stochastic interpolants' These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way.
  arXiv  Detail & Related papers  (2023-03-15T17:43:42Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Probability flow solution of the Fokker-Planck equation [10.484851004093919]
  We introduce an alternative scheme based on integrating an ordinary differential equation that describes the flow of probability. Unlike the dynamics, this equation deterministically pushes samples from the initial density onto samples from the solution at any later time. Our approach is based on recent advances in score-based diffusion for generative modeling.
  arXiv  Detail & Related papers  (2022-06-09T17:37:09Z)
• A blob method method for inhomogeneous diffusion with applications to multi-agent control and sampling [0.6562256987706128]
  We develop a deterministic particle method for the weighted porous medium equation (WPME) and prove its convergence on bounded time intervals. Our method has natural applications to multi-agent coverage algorithms and sampling probability measures.
  arXiv  Detail & Related papers  (2022-02-25T19:49:05Z)
• Moment evolution equations and moment matching for stochastic image EPDiff [68.97335984455059]
  Models of image deformation allow study of time-continuous effects transforming images by deforming the image domain. Applications include medical image analysis with both population trends and random subject specific variation. We use moment approximations of the corresponding Ito diffusion to construct estimators for statistical inference in the parameters full model.
  arXiv  Detail & Related papers  (2021-10-07T11:08:11Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.