Stochastic quantization and diffusion models

• URL: http://arxiv.org/abs/2411.11297v1
• Date: Mon, 18 Nov 2024 05:47:41 GMT
• Title: Stochastic quantization and diffusion models
• Authors: Kenji Fukushima, Syo Kamata,
• Abstract summary: This is a review of the possible connection between the quantization in physics and the diffusion models in machine learning. For machine-learning applications, the denoising diffusion model has been established as a successful technique. In this review, we focus on an SDE approach used in the score-based generative modeling.
• Score: 0.0
• License:
• Abstract: This is a pedagogical review of the possible connection between the stochastic quantization in physics and the diffusion models in machine learning. For machine-learning applications, the denoising diffusion model has been established as a successful technique, which is formulated in terms of the stochastic differential equation (SDE). In this review, we focus on an SDE approach used in the score-based generative modeling. Interestingly, the evolution of the probability distribution is equivalently described by a particular class of SDEs, and in a particular limit, the stochastic noises can be eliminated. Then, we turn to a similar mathematical formulation in quantum physics, that is, the stochastic quantization. We make a brief overview on the stochastic quantization using a simple toy model of the one-dimensional integration. The analogy between the diffusion model and the stochastic quantization is clearly seen in this concrete example. Finally, we discuss how the sign problem arises in the toy model with complex parameters. The origin of the difficulty is understood based on the Lefschetz thimble analysis. We point out that the SDE is not invariant under the variable change which induces a kernel and a special choice of the kernel guided by the Lefschetz thimble analysis can reduce the sign problem.

Related papers

• Understanding Diffusion Models by Feynman's Path Integral [2.4373900721120285]
  We introduce a novel formulation of diffusion models using Feynman's integral path. We find this formulation providing comprehensive descriptions of score-based generative models. We also demonstrate the derivation of backward differential equations and loss functions.
  arXiv  Detail & Related papers  (2024-03-17T16:24:29Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Causal Modeling with Stationary Diffusions [89.94899196106223]
  We learn differential equations whose stationary densities model a system's behavior under interventions. We show that they generalize to unseen interventions on their variables, often better than classical approaches. Our inference method is based on a new theoretical result that expresses a stationarity condition on the diffusion's generator in a reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2023-10-26T14:01:17Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Latent SDEs on Homogeneous Spaces [9.361372513858043]
  We consider the problem of variational Bayesian inference in a latent variable model where a (possibly complex) observed geometric process is governed by the solution of a latent differential equation (SDE) Experiments demonstrate that a latent SDE of the proposed type can be learned efficiently by means of an existing one-step Euler-Maruyama scheme.
  arXiv  Detail & Related papers  (2023-06-28T14:18:52Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Score-based Generative Modeling Through Backward Stochastic Differential Equations: Inversion and Generation [6.2255027793924285]
  The proposed BSDE-based diffusion model represents a novel approach to diffusion modeling, which extends the application of differential equations (SDEs) in machine learning. We demonstrate the theoretical guarantees of the model, the benefits of using Lipschitz networks for score matching, and its potential applications in various areas such as diffusion inversion, conditional diffusion, and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-04-26T01:15:35Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Stochastic Differential Equations with Variational Wishart Diffusions [18.590352916158093]
  We present a non-parametric way of inferring differential equations for both regression tasks and continuous-time dynamical modelling. The work has high emphasis on the part of the differential equation, also known as the diffusion, and modelling it by means of Wishart processes.
  arXiv  Detail & Related papers  (2020-06-26T10:21:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.