Particle Denoising Diffusion Sampler

• URL: http://arxiv.org/abs/2402.06320v2
• Date: Sat, 15 Jun 2024 11:28:05 GMT
• Title: Particle Denoising Diffusion Sampler
• Authors: Angus Phillips, Hai-Dang Dau, Michael John Hutchinson, Valentin De Bortoli, George Deligiannidis, Arnaud Doucet,
• Abstract summary: Particle Denoising Diffusion Sampler (PDDS) provides consistent estimates under mild assumptions. We demonstrate PDDS on multimodal and high dimensional sampling tasks.
• Score: 32.310922004771776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Denoising diffusion models have become ubiquitous for generative modeling. The core idea is to transport the data distribution to a Gaussian by using a diffusion. Approximate samples from the data distribution are then obtained by estimating the time-reversal of this diffusion using score matching ideas. We follow here a similar strategy to sample from unnormalized probability densities and compute their normalizing constants. However, the time-reversed diffusion is here simulated by using an original iterative particle scheme relying on a novel score matching loss. Contrary to standard denoising diffusion models, the resulting Particle Denoising Diffusion Sampler (PDDS) provides asymptotically consistent estimates under mild assumptions. We demonstrate PDDS on multimodal and high dimensional sampling tasks.

Related papers

• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers. We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Boosting Diffusion Models with Moving Average Sampling in Frequency Domain [101.43824674873508]
  Diffusion models rely on the current sample to denoise the next one, possibly resulting in denoising instability. In this paper, we reinterpret the iterative denoising process as model optimization and leverage a moving average mechanism to ensemble all the prior samples. We name the complete approach "Moving Average Sampling in Frequency domain (MASF)"
  arXiv  Detail & Related papers  (2024-03-26T16:57:55Z)
• Lecture Notes in Probabilistic Diffusion Models [0.5361320134021585]
  Diffusion models are loosely modelled based on non-equilibrium thermodynamics. The diffusion model learns the data manifold to which the original and thus the reconstructed data samples belong. Diffusion models have -- unlike variational autoencoder and flow models -- latent variables with the same dimensionality as the original data.
  arXiv  Detail & Related papers  (2023-12-16T09:36:54Z)
• Projection Regret: Reducing Background Bias for Novelty Detection via Diffusion Models [72.07462371883501]
  We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information. PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality. Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
  arXiv  Detail & Related papers  (2023-12-05T09:44:47Z)
• Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Models [76.46246743508651]
  We show that current diffusion models actually have an expressive bottleneck in backward denoising. We introduce soft mixture denoising (SMD), an expressive and efficient model for backward denoising.
  arXiv  Detail & Related papers  (2023-09-25T12:03:32Z)
• Semi-Implicit Denoising Diffusion Models (SIDDMs) [50.30163684539586]
  Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. We introduce a novel approach that tackles the problem by matching implicit and explicit factors. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps.
  arXiv  Detail & Related papers  (2023-06-21T18:49:22Z)
• Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces [0.0]
  We develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process. As an example, we introduce Blackout Diffusion'', which learns to produce samples from an empty image instead of from noise.
  arXiv  Detail & Related papers  (2023-05-18T16:24:12Z)
• To smooth a cloud or to pin it down: Guarantees and Insights on Score Matching in Denoising Diffusion Models [20.315727650065007]
  Denoising diffusion models are a class of generative models which have recently achieved state-of-the-art results across many domains. We leverage known connections to control akin to the F"ollmer drift to extend established neural network approximation results for the F"ollmer drift to denoising diffusion models and samplers.
  arXiv  Detail & Related papers  (2023-05-16T16:56:19Z)
• Denoising Diffusion Samplers [41.796349001299156]
  Denoising diffusion models are a popular class of generative models providing state-of-the-art results in many domains. We explore a similar idea to sample approximately from unnormalized probability density functions and estimate their normalizing constants. While score matching is not applicable in this context, we can leverage many of the ideas introduced in generative modeling for Monte Carlo sampling.
  arXiv  Detail & Related papers  (2023-02-27T14:37:16Z)
• From Denoising Diffusions to Denoising Markov Models [38.33676858989955]
  Denoising diffusions are state-of-the-art generative models exhibiting remarkable empirical performance. We propose a unifying framework generalising this approach to a wide class of spaces and leading to an original extension of score matching.
  arXiv  Detail & Related papers  (2022-11-07T14:34:27Z)
• Generative Modeling with Denoising Auto-Encoders and Langevin Sampling [88.83704353627554]
  We show that both DAE and DSM provide estimates of the score of the smoothed population density. We then apply our results to the homotopy method of arXiv:1907.05600 and provide theoretical justification for its empirical success.
  arXiv  Detail & Related papers  (2020-01-31T23:50:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.