Denoising MCMC for Accelerating Diffusion-Based Generative Models

• URL: http://arxiv.org/abs/2209.14593v1
• Date: Thu, 29 Sep 2022 07:16:10 GMT
• Title: Denoising MCMC for Accelerating Diffusion-Based Generative Models
• Authors: Beomsu Kim and Jong Chul Ye
• Abstract summary: Diffusion models are powerful generative models that simulate the reverse of diffusion processes using score functions to synthesize data from noise. Here, we propose an approach to accelerating score-based sampling: Denoising MCMC. We show that Denoising Langevin Gibbs (DLG), an instance of DMCMC, successfully accelerates all six reverse-S/ODE computation tasks.
• Score: 54.06799491319278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models are powerful generative models that simulate the reverse of diffusion processes using score functions to synthesize data from noise. The sampling process of diffusion models can be interpreted as solving the reverse stochastic differential equation (SDE) or the ordinary differential equation (ODE) of the diffusion process, which often requires up to thousands of discretization steps to generate a single image. This has sparked a great interest in developing efficient integration techniques for reverse-S/ODEs. Here, we propose an orthogonal approach to accelerating score-based sampling: Denoising MCMC (DMCMC). DMCMC first uses MCMC to produce samples in the product space of data and variance (or diffusion time). Then, a reverse-S/ODE integrator is used to denoise the MCMC samples. Since MCMC traverses close to the data manifold, the computation cost of producing a clean sample for DMCMC is much less than that of producing a clean sample from noise. To verify the proposed concept, we show that Denoising Langevin Gibbs (DLG), an instance of DMCMC, successfully accelerates all six reverse-S/ODE integrators considered in this work on the tasks of CIFAR10 and CelebA-HQ-256 image generation. Notably, combined with integrators of Karras et al. (2022) and pre-trained score models of Song et al. (2021b), DLG achieves SOTA results. In the limited number of score function evaluation (NFE) settings on CIFAR10, we have $3.86$ FID with $\approx 10$ NFE and $2.63$ FID with $\approx 20$ NFE. On CelebA-HQ-256, we have $6.99$ FID with $\approx 160$ NFE, which beats the current best record of Kim et al. (2022) among score-based models, $7.16$ FID with $4000$ NFE. Code: https://github.com/1202kbs/DMCMC

Related papers

• Optimizing Noise Schedules of Generative Models in High Dimensionss [18.19470017419402]
  We show that noise schedules specific to preserving variance (VP) and variance exploding (VE) allow for the recovery of both high- and low-level features. We also show that these schedules yield generative models for the GM and Curie-Weiss (CW) model whose probability flow ODE can be discretized.
  arXiv  Detail & Related papers  (2025-01-02T00:39:00Z)
• EMC$^2$: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence [43.96096434967746]
  A key challenge in contrastive learning is to generate negative samples from a large sample set to contrast with positive samples. We propose an Efficient Markov Chain Monte Carlo negative sampling method for Contrastive learning (EMC$2$) We prove that EMC$2$ is the first algorithm that exhibits global convergence (to stationarity) regardless of the choice of batch size.
  arXiv  Detail & Related papers  (2024-04-16T13:53:58Z)
• Improved DDIM Sampling with Moment Matching Gaussian Mixtures [1.450405446885067]
  We propose using a Gaussian Mixture Model (GMM) as reverse transition operator ( kernel) within the Denoising Diffusion Implicit Models (DDIM) framework. We match the first and second order central moments of the DDPM forward marginals by constraining the parameters of the GMM. Our results suggest that using the GMM kernel leads to significant improvements in the quality of the generated samples when the number of sampling steps is small.
  arXiv  Detail & Related papers  (2023-11-08T00:24:50Z)
• Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion [56.38386580040991]
  Consistency Trajectory Model (CTM) is a generalization of Consistency Models (CM) CTM enables the efficient combination of adversarial training and denoising score matching loss to enhance performance. Unlike CM, CTM's access to the score function can streamline the adoption of established controllable/conditional generation methods.
  arXiv  Detail & Related papers  (2023-10-01T05:07:17Z)
• Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models [49.81937966106691]
  We develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach.
  arXiv  Detail & Related papers  (2023-06-15T16:30:08Z)
• Pseudo Numerical Methods for Diffusion Models on Manifolds [77.40343577960712]
  Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples. DDPMs require hundreds to thousands of iterations to produce final samples. We propose pseudo numerical methods for diffusion models (PNDMs) PNDMs can generate higher quality synthetic images with only 50 steps compared with 1000-step DDIMs (20x speedup)
  arXiv  Detail & Related papers  (2022-02-20T10:37:52Z)
• Score-Based Generative Modeling through Stochastic Differential Equations [114.39209003111723]
  We present a differential equation that transforms a complex data distribution to a known prior distribution by injecting noise. A corresponding reverse-time SDE transforms the prior distribution back into the data distribution by slowly removing the noise. By leveraging advances in score-based generative modeling, we can accurately estimate these scores with neural networks. We demonstrate high fidelity generation of 1024 x 1024 images for the first time from a score-based generative model.
  arXiv  Detail & Related papers  (2020-11-26T19:39:10Z)
• Denoising Diffusion Implicit Models [117.03720513930335]
  We present denoising diffusion implicit models (DDIMs) for iterative implicit probabilistic models with the same training procedure as DDPMs. DDIMs can produce high quality samples $10 times$ to $50 times$ faster in terms of wall-clock time compared to DDPMs.
  arXiv  Detail & Related papers  (2020-10-06T06:15:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.