A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models

• URL: http://arxiv.org/abs/2312.07243v1
• Date: Tue, 12 Dec 2023 13:19:40 GMT
• Title: A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models
• Authors: Enshu Liu, Xuefei Ning, Huazhong Yang, Yu Wang
• Abstract summary: In this paper, we propose a unified sampling framework (USF) to study the optional strategies for solver. Under this framework, we reveal that taking different solving strategies at different timesteps may help further decrease the truncation error. We demonstrate that $S3$ can find outstanding solver schedules which outperform the state-of-the-art sampling methods.
• Score: 21.305868355976394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent years have witnessed the rapid progress and broad application of diffusion probabilistic models (DPMs). Sampling from DPMs can be viewed as solving an ordinary differential equation (ODE). Despite the promising performance, the generation of DPMs usually consumes much time due to the large number of function evaluations (NFE). Though recent works have accelerated the sampling to around 20 steps with high-order solvers, the sample quality with less than 10 NFE can still be improved. In this paper, we propose a unified sampling framework (USF) to study the optional strategies for solver. Under this framework, we further reveal that taking different solving strategies at different timesteps may help further decrease the truncation error, and a carefully designed \emph{solver schedule} has the potential to improve the sample quality by a large margin. Therefore, we propose a new sampling framework based on the exponential integral formulation that allows free choices of solver strategy at each step and design specific decisions for the framework. Moreover, we propose $S^3$, a predictor-based search method that automatically optimizes the solver schedule to get a better time-quality trade-off of sampling. We demonstrate that $S^3$ can find outstanding solver schedules which outperform the state-of-the-art sampling methods on CIFAR-10, CelebA, ImageNet, and LSUN-Bedroom datasets. Specifically, we achieve 2.69 FID with 10 NFE and 6.86 FID with 5 NFE on CIFAR-10 dataset, outperforming the SOTA method significantly. We further apply $S^3$ to Stable-Diffusion model and get an acceleration ratio of 2$\times$, showing the feasibility of sampling in very few steps without retraining the neural network.

Related papers

• DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation [68.55191764622525]
  Diffusion models (DPMs) have shown remarkable performance in visual synthesis but are computationally expensive due to the need for multiple evaluations during the sampling. Recent predictor synthesis-or diffusion samplers have significantly reduced the required number of evaluations, but inherently suffer from a misalignment issue. We introduce a new fast DPM sampler called DC-CPRr, which leverages dynamic compensation to mitigate the misalignment.
  arXiv  Detail & Related papers  (2024-09-05T17:59:46Z)
• Accelerating Diffusion Sampling with Optimized Time Steps [69.21208434350567]
  Diffusion probabilistic models (DPMs) have shown remarkable performance in high-resolution image synthesis. Their sampling efficiency is still to be desired due to the typically large number of sampling steps. Recent advancements in high-order numerical ODE solvers for DPMs have enabled the generation of high-quality images with much fewer sampling steps.
  arXiv  Detail & Related papers  (2024-02-27T10:13:30Z)
• S$^{2}$-DMs:Skip-Step Diffusion Models [10.269647566864247]
  Diffusion models have emerged as powerful generative tools, rivaling GANs in sample quality and mirroring the likelihood scores of autoregressive models. A subset of these models, exemplified by DDIMs, exhibit an inherent asymmetry: they are trained over $T$ steps but only sample from a subset of $T$ during generation. This selective sampling approach, though optimized for speed, inadvertently misses out on vital information from the unsampled steps, leading to potential compromises in sample quality. We present the S$2$-DMs, which is a new training method by using an innovative $L
  arXiv  Detail & Related papers  (2024-01-03T03:08:32Z)
• Sampler Scheduler for Diffusion Models [0.0]
  Diffusion modeling (DM) has high-quality generative performance. Currently, there is a contradiction in samplers for diffusion-based generative models. We propose the feasibility of using different samplers (ODE/SDE) on different sampling steps of the same sampling process.
  arXiv  Detail & Related papers  (2023-11-12T13:35:25Z)
• Optimal Budgeted Rejection Sampling for Generative Models [54.050498411883495]
  Rejection sampling methods have been proposed to improve the performance of discriminator-based generative models. We first propose an Optimal Budgeted Rejection Sampling scheme that is provably optimal. Second, we propose an end-to-end method that incorporates the sampling scheme into the training procedure to further enhance the model's overall performance.
  arXiv  Detail & Related papers  (2023-11-01T11:52:41Z)
• DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics [23.030972042695275]
  Diffusion models (DPMs) have exhibited excellent performance for high-fidelity image generation while suffering from inefficient sampling. Recent works accelerate the sampling procedure by proposing fast ODE solvers that leverage the specific ODE form of DPMs. We propose a novel formulation towards the optimal parameterization during sampling that minimizes the first-order discretization error.
  arXiv  Detail & Related papers  (2023-10-20T04:23:12Z)
• SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models [66.67616086310662]
  Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed. We propose SA-of-r, which is an improved efficient Adams method for solving diffusion SDE to generate data with high quality.
  arXiv  Detail & Related papers  (2023-09-10T12:44:54Z)
• Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps [23.144083737873263]
  Diffusion Probabilistic Models (DPM) have shown remarkable efficacy in the synthesis of high-quality images. Previous work has attempted to mitigate this issue by perturbing inputs during training. We propose a novel sampling method that we propose, without retraining the model.
  arXiv  Detail & Related papers  (2023-05-24T21:39:27Z)
• Preconditioned Score-based Generative Models [49.88840603798831]
  An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We propose a model-agnostic bfem preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining.
  arXiv  Detail & Related papers  (2023-02-13T16:30:53Z)
• UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models [92.43617471204963]
  Diffusion probabilistic models (DPMs) have demonstrated a very promising ability in high-resolution image synthesis. We develop a unified corrector (UniC) that can be applied after any existing DPM sampler to increase the order of accuracy. We propose a unified predictor-corrector framework called UniPC for the fast sampling of DPMs.
  arXiv  Detail & Related papers  (2023-02-09T18:59:48Z)
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps [45.612477740555406]
  Diffusion probabilistic models (DPMs) are emerging powerful generative models. DPM-r is suitable for both discrete-time and continuous-time DPMs without any further training. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset.
  arXiv  Detail & Related papers  (2022-06-02T08:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.