Fast Sampling of Diffusion Models with Exponential Integrator

• URL: http://arxiv.org/abs/2204.13902v1
• Date: Fri, 29 Apr 2022 06:32:38 GMT
• Title: Fast Sampling of Diffusion Models with Exponential Integrator
• Authors: Qinsheng Zhang, Yongxin Chen
• Abstract summary: We propose a fast sampling method for DMs with much less number of steps while retaining high sample quality. The proposed method can be applied to any DMs and can generate high-fidelity samples in as few as 10 steps.
• Score: 9.467521554542273
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The past few years have witnessed the great success of Diffusion models~(DMs) in generating high-fidelity samples in generative modeling tasks. A major limitation of the DM is its notoriously slow sampling procedure which normally requires hundreds to thousands of time discretization steps of the learned diffusion process to reach the desired accuracy. Our goal is to develop a fast sampling method for DMs with much less number of steps while retaining high sample quality. To this end, we systematically analyze the sampling procedure in DMs and identify key factors that affect the sample quality, among which the method of discretization is most crucial. By carefully examining the learned diffusion process, we propose Diffusion Exponential Integrator Sampler~(DEIS). It is based on the Exponential Integrator designed for discretizing ordinary differential equations (ODEs) and leverages a semilinear structure of the learned diffusion process to reduce the discretization error. The proposed method can be applied to any DMs and can generate high-fidelity samples in as few as 10 steps. In our experiments, it takes about 3 minutes on one A6000 GPU to generate $50k$ images from CIFAR10. Moreover, by directly using pre-trained DMs, we achieve the state-of-art sampling performance when the number of score function evaluation~(NFE) is limited, e.g., 3.37 FID and 9.74 Inception score with only 15 NFEs on CIFAR10.

Related papers

• Align Your Steps: Optimizing Sampling Schedules in Diffusion Models [63.927438959502226]
  Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed, relying on many sequential function evaluations through large neural networks. We propose a general and principled approach to optimizing the sampling schedules of DMs for high-quality outputs.
  arXiv  Detail & Related papers  (2024-04-22T18:18:41Z)
• Fast Diffusion Model [122.36693015093041]
  Diffusion models (DMs) have been adopted across diverse fields with their abilities in capturing intricate data distributions. In this paper, we propose a Fast Diffusion Model (FDM) to significantly speed up DMs from a DM optimization perspective.
  arXiv  Detail & Related papers  (2023-06-12T09:38:04Z)
• Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps [23.144083737873263]
  Diffusion Probabilistic Models (DPM) have shown remarkable efficacy in the synthesis of high-quality images. Previous work has attempted to mitigate this issue by perturbing inputs during training. We propose a novel sampling method that we propose, without retraining the model.
  arXiv  Detail & Related papers  (2023-05-24T21:39:27Z)
• Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis [26.907301901503835]
  Denoising diffusion probabilistic models (DDPMs) are a class of powerful generative models. DDPMs generally need hundreds or thousands of sequential function evaluations (steps) of neural networks to generate a sample. This paper aims to develop a fast sampling method for DDPMs requiring much fewer steps while retaining high sample quality.
  arXiv  Detail & Related papers  (2023-04-22T16:58:47Z)
• Preconditioned Score-based Generative Models [49.88840603798831]
  An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We propose a model-agnostic bfem preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining.
  arXiv  Detail & Related papers  (2023-02-13T16:30:53Z)
• Fast Inference in Denoising Diffusion Models via MMD Finetuning [23.779985842891705]
  We present MMD-DDM, a novel method for fast sampling of diffusion models. Our approach is based on the idea of using the Maximum Mean Discrepancy (MMD) to finetune the learned distribution with a given budget of timesteps. Our findings show that the proposed method is able to produce high-quality samples in a fraction of the time required by widely-used diffusion models.
  arXiv  Detail & Related papers  (2023-01-19T09:48:07Z)
• Fast Sampling of Diffusion Models via Operator Learning [74.37531458470086]
  We use neural operators, an efficient method to solve the probability flow differential equations, to accelerate the sampling process of diffusion models. Compared to other fast sampling methods that have a sequential nature, we are the first to propose a parallel decoding method. We show our method achieves state-of-the-art FID of 3.78 for CIFAR-10 and 7.83 for ImageNet-64 in the one-model-evaluation setting.
  arXiv  Detail & Related papers  (2022-11-24T07:30:27Z)
• Pseudo Numerical Methods for Diffusion Models on Manifolds [77.40343577960712]
  Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples. DDPMs require hundreds to thousands of iterations to produce final samples. We propose pseudo numerical methods for diffusion models (PNDMs) PNDMs can generate higher quality synthetic images with only 50 steps compared with 1000-step DDIMs (20x speedup)
  arXiv  Detail & Related papers  (2022-02-20T10:37:52Z)
• Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality [44.37533757879762]
  We introduce Differentiable Diffusion Sampler Search (DDSS), a method that optimize fast samplers for any pre-trained diffusion model. We also present Generalized Gaussian Diffusion Models (GGDM), a family of flexible non-Markovian samplers for diffusion models. Our method is compatible with any pre-trained diffusion model without fine-tuning or re-training required.
  arXiv  Detail & Related papers  (2022-02-11T18:53:18Z)
• Denoising Diffusion Implicit Models [117.03720513930335]
  We present denoising diffusion implicit models (DDIMs) for iterative implicit probabilistic models with the same training procedure as DDPMs. DDIMs can produce high quality samples $10 times$ to $50 times$ faster in terms of wall-clock time compared to DDPMs.
  arXiv  Detail & Related papers  (2020-10-06T06:15:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.