Sampler Scheduler for Diffusion Models

• URL: http://arxiv.org/abs/2311.06845v1
• Date: Sun, 12 Nov 2023 13:35:25 GMT
• Title: Sampler Scheduler for Diffusion Models
• Authors: Zitong Cheng
• Abstract summary: Diffusion modeling (DM) has high-quality generative performance. Currently, there is a contradiction in samplers for diffusion-based generative models. We propose the feasibility of using different samplers (ODE/SDE) on different sampling steps of the same sampling process.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion modeling (DM) has high-quality generative performance, and the sampling problem is an important part of the DM performance. Thanks to efficient differential equation solvers, the sampling speed can be reduced while higher sampling quality is guaranteed. However, currently, there is a contradiction in samplers for diffusion-based generative models: the mainstream sampler choices are diverse, each with its own characteristics in terms of performance. However, only a single sampler algorithm can be specified on all sampling steps in the generative process. This often makes one torn between sampler choices; in other words, it makes it difficult to fully utilize the advantages of each sampler. In this paper, we propose the feasibility of using different samplers (ODE/SDE) on different sampling steps of the same sampling process based on analyzing and generalizing the updating formulas of each mainstream sampler, and experimentally demonstrate that such a multi-sampler scheduling improves the sampling results to some extent. In particular, we also verify that the combination of using SDE in the early sampling steps and ODE in the later sampling steps solves the inherent problems previously caused by using both singly. We show that our design changes improve the sampling efficiency and quality in previous work. For instance, when Number of Function Evaluations (NFE) = 24, the ODE Sampler Scheduler achieves a FID score of 1.91 on the CIFAR-10 dataset, compared to 2.02 for DPM++ 2M, 1.97 for DPM2, and 11.90 for Heun for the same NFE. Meanwhile the Sampler Scheduler with the combined scheduling of SDE and ODE reaches 1.899, compared to 18.63 for Euler a, 3.14 for DPM2 a and 23.14 for DPM++ SDE.

Related papers

• DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation [68.55191764622525]
  Diffusion models (DPMs) have shown remarkable performance in visual synthesis but are computationally expensive due to the need for multiple evaluations during the sampling. Recent predictor synthesis-or diffusion samplers have significantly reduced the required number of evaluations, but inherently suffer from a misalignment issue. We introduce a new fast DPM sampler called DC-CPRr, which leverages dynamic compensation to mitigate the misalignment.
  arXiv  Detail & Related papers  (2024-09-05T17:59:46Z)
• Accelerating Diffusion Sampling with Optimized Time Steps [69.21208434350567]
  Diffusion probabilistic models (DPMs) have shown remarkable performance in high-resolution image synthesis. Their sampling efficiency is still to be desired due to the typically large number of sampling steps. Recent advancements in high-order numerical ODE solvers for DPMs have enabled the generation of high-quality images with much fewer sampling steps.
  arXiv  Detail & Related papers  (2024-02-27T10:13:30Z)
• Towards Fast Stochastic Sampling in Diffusion Generative Models [22.01769257075573]
  Diffusion models suffer from slow sample generation at inference time. We propose Splittings for fast sampling in pre-trained diffusion models in augmented spaces. We show that a naive application of splitting is sub-optimal for fast sampling.
  arXiv  Detail & Related papers  (2024-02-11T14:04:13Z)
• SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models [66.67616086310662]
  Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed. We propose SA-of-r, which is an improved efficient Adams method for solving diffusion SDE to generate data with high quality.
  arXiv  Detail & Related papers  (2023-09-10T12:44:54Z)
• Restart Sampling for Improving Generative Processes [30.745245429072735]
  ODE-based samplers are fast but plateau in performance while SDE-based samplers deliver higher sample quality at the cost of increased sampling time. We propose a novel sampling algorithm called Restart in order to better balance discretization errors and contraction.
  arXiv  Detail & Related papers  (2023-06-26T17:48:25Z)
• Parallel Sampling of Diffusion Models [76.3124029406809]
  Diffusion models are powerful generative models but suffer from slow sampling. We present ParaDiGMS, a novel method to accelerate the sampling of pretrained diffusion models by denoising multiple steps in parallel.
  arXiv  Detail & Related papers  (2023-05-25T17:59:42Z)
• Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis [26.907301901503835]
  Denoising diffusion probabilistic models (DDPMs) are a class of powerful generative models. DDPMs generally need hundreds or thousands of sequential function evaluations (steps) of neural networks to generate a sample. This paper aims to develop a fast sampling method for DDPMs requiring much fewer steps while retaining high sample quality.
  arXiv  Detail & Related papers  (2023-04-22T16:58:47Z)
• Preconditioned Score-based Generative Models [49.88840603798831]
  An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We propose a model-agnostic bfem preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining.
  arXiv  Detail & Related papers  (2023-02-13T16:30:53Z)
• UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models [92.43617471204963]
  Diffusion probabilistic models (DPMs) have demonstrated a very promising ability in high-resolution image synthesis. We develop a unified corrector (UniC) that can be applied after any existing DPM sampler to increase the order of accuracy. We propose a unified predictor-corrector framework called UniPC for the fast sampling of DPMs.
  arXiv  Detail & Related papers  (2023-02-09T18:59:48Z)
• DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models [45.612477740555406]
  We propose DPM-r++, a high-order solver for guided sampling of DPMs. We show that DPM-r++ can generate high-quality samples within only 15 to 20 steps for guided sampling by pixel-space and latent-space DPMs.
  arXiv  Detail & Related papers  (2022-11-02T13:14:30Z)
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps [45.612477740555406]
  Diffusion probabilistic models (DPMs) are emerging powerful generative models. DPM-r is suitable for both discrete-time and continuous-time DPMs without any further training. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset.
  arXiv  Detail & Related papers  (2022-06-02T08:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.