Diffusion-PINN Sampler

• URL: http://arxiv.org/abs/2410.15336v1
• Date: Sun, 20 Oct 2024 09:02:16 GMT
• Title: Diffusion-PINN Sampler
• Authors: Zhekun Shi, Longlin Yu, Tianyu Xie, Cheng Zhang,
• Abstract summary: We introduce a novel diffusion-based sampling algorithm that estimates the drift term by solving the governing partial differential equation of the log-density of the underlying SDE marginals via physics-informed neural networks (PINN) We prove that the error of log-density approximation can be controlled by the PINN residual loss, enabling us to establish convergence guarantees of DPS.
• Score: 6.656265182236135
• License:
• Abstract: Recent success of diffusion models has inspired a surge of interest in developing sampling techniques using reverse diffusion processes. However, accurately estimating the drift term in the reverse stochastic differential equation (SDE) solely from the unnormalized target density poses significant challenges, hindering existing methods from achieving state-of-the-art performance. In this paper, we introduce the Diffusion-PINN Sampler (DPS), a novel diffusion-based sampling algorithm that estimates the drift term by solving the governing partial differential equation of the log-density of the underlying SDE marginals via physics-informed neural networks (PINN). We prove that the error of log-density approximation can be controlled by the PINN residual loss, enabling us to establish convergence guarantees of DPS. Experiments on a variety of sampling tasks demonstrate the effectiveness of our approach, particularly in accurately identifying mixing proportions when the target contains isolated components.

Related papers

• A hybrid FEM-PINN method for time-dependent partial differential equations [9.631238071993282]
  We present a hybrid numerical method for solving evolution differential equations (PDEs) by merging the time finite element method with deep neural networks. The advantages of such a hybrid formulation are twofold: statistical errors are avoided for the integral in the time direction, and the neural network's output can be regarded as a set of reduced spatial basis functions.
  arXiv  Detail & Related papers  (2024-09-04T15:28:25Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference [6.37512592611305]
  In deep Gaussian processes (DGPs) a sparse integration location called inducing points are selected to approximate the posterior distribution of the model. Traditional variational inference approaches to posterior approximation often lead to significant bias. We propose an alternative method called Denoising Diffusion Variational Inference (DDVI) that uses a denoising diffusion differential equation (SDE) to generate posterior samples of inducing variables.
  arXiv  Detail & Related papers  (2024-07-24T06:39:58Z)
• Repulsive Latent Score Distillation for Solving Inverse Problems [31.255943277671893]
  Score Distillation Sampling (SDS) has been pivotal for leveraging pre-trained diffusion models in downstream tasks such as inverse problems. We introduce a novel variational framework for posterior sampling to address mode collapse and latent space inversion. We extend this framework with an augmented variational distribution that disentangles the latent and data.
  arXiv  Detail & Related papers  (2024-06-24T14:43:02Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Semi-Implicit Denoising Diffusion Models (SIDDMs) [50.30163684539586]
  Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. We introduce a novel approach that tackles the problem by matching implicit and explicit factors. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps.
  arXiv  Detail & Related papers  (2023-06-21T18:49:22Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Models [25.468628820118564]
  Low sampling efficiency of diffusion probabilistic models (DDPMs) limits further applications. We construct an error-robust Adams solver (ERA-r) Experiments on Cifar10, LSUN-Church, and LSUN-Bedroom demonstrate our proposed ERA-r 5.14, 9.42, and 9.69 Fenchel Inception Distance (FID) for image generation, with only 10 network evaluations.
  arXiv  Detail & Related papers  (2023-01-30T14:32:47Z)
• Stationary Density Estimation of It\^o Diffusions Using Deep Learning [6.8342505943533345]
  We consider the density estimation problem associated with the stationary measure of ergodic Ito diffusions from a discrete-time series. We employ deep neural networks to approximate the drift and diffusion terms of the SDE. We establish the convergence of the proposed scheme under appropriate mathematical assumptions.
  arXiv  Detail & Related papers  (2021-09-09T01:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.