SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models

• URL: http://arxiv.org/abs/2309.05019v3
• Date: Tue, 24 Jun 2025 18:47:02 GMT
• Title: SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models
• Authors: Shuchen Xue, Mingyang Yi, Weijian Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, Zhi-Ming Ma,
• Abstract summary: Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks.<n>As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed.<n>We propose textitSA-r, which is an improved efficient method for solving SDE to generate data with high quality.
• Score: 63.49229402384349
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed. The majority of such techniques consider solving the diffusion ODE due to its superior efficiency. However, stochastic sampling could offer additional advantages in generating diverse and high-quality data. In this work, we engage in a comprehensive analysis of stochastic sampling from two aspects: variance-controlled diffusion SDE and linear multi-step SDE solver. Based on our analysis, we propose \textit{SA-Solver}, which is an improved efficient stochastic Adams method for solving diffusion SDE to generate data with high quality. Our experiments show that \textit{SA-Solver} achieves: 1) improved or comparable performance compared with the existing state-of-the-art (SOTA) sampling methods for few-step sampling; 2) SOTA FID on substantial benchmark datasets under a suitable number of function evaluations (NFEs). Code is available at https://github.com/scxue/SA-Solver.

Related papers

• Beyond Scores: Proximal Diffusion Models [10.27283386401996]
  We develop Proximal Diffusion Models (ProxDM) to learn proximal operators of the log-density.<n>We show that two variants of ProxDM achieve significantly faster within just a few sampling steps compared to conventional score-matching methods.
  arXiv  Detail & Related papers  (2025-07-11T18:30:09Z)
• Repulsive Latent Score Distillation for Solving Inverse Problems [31.255943277671893]
  Score Distillation Sampling (SDS) has been pivotal for leveraging pre-trained diffusion models in downstream tasks such as inverse problems. We introduce a novel variational framework for posterior sampling to address mode collapse and latent space inversion. We extend this framework with an augmented variational distribution that disentangles the latent and data.
  arXiv  Detail & Related papers  (2024-06-24T14:43:02Z)
• AdjointDEIS: Efficient Gradients for Diffusion Models [2.0795007613453445]
  We show that continuous adjoint equations for diffusion SDEs actually simplify to a simple ODE.<n>We also demonstrate the effectiveness of AdjointDEIS for guided generation with an adversarial attack in the form of the face morphing problem.
  arXiv  Detail & Related papers  (2024-05-23T19:51:33Z)
• Iterated Denoising Energy Matching for Sampling from Boltzmann Densities [109.23137009609519]
  Iterated Denoising Energy Matching (iDEM) iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our matching objective. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5times$ faster.
  arXiv  Detail & Related papers  (2024-02-09T01:11:23Z)
• Risk-Sensitive Diffusion: Robustly Optimizing Diffusion Models with Noisy Samples [58.68233326265417]
  Non-image data are prevalent in real applications and tend to be noisy. Risk-sensitive SDE is a type of differential equation (SDE) parameterized by the risk vector. We conduct systematic studies for both Gaussian and non-Gaussian noise distributions.
  arXiv  Detail & Related papers  (2024-02-03T08:41:51Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Elucidating the solution space of extended reverse-time SDE for diffusion models [54.23536653351234]
  Diffusion models (DMs) demonstrate potent image generation capabilities in various generative modeling tasks. Their primary limitation lies in slow sampling speed, requiring hundreds or thousands of sequential function evaluations to generate high-quality images. We formulate the sampling process as an extended reverse-time SDE, unifying prior explorations into ODEs and SDEs. We devise fast and training-free samplers, ER-SDE-rs, achieving state-of-the-art performance across all samplers.
  arXiv  Detail & Related papers  (2023-09-12T12:27:17Z)
• Fast Diffusion Model [122.36693015093041]
  Diffusion models (DMs) have been adopted across diverse fields with their abilities in capturing intricate data distributions. In this paper, we propose a Fast Diffusion Model (FDM) to significantly speed up DMs from a DM optimization perspective.
  arXiv  Detail & Related papers  (2023-06-12T09:38:04Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution [82.50210340928173]
  randomness of diffusion models results in ineffectiveness and instability, making it challenging for users to guarantee the quality of SR results. We propose a plug-and-play sampling method that owns the potential to benefit a series of diffusion-based SR methods. The quality of SR results sampled by the proposed method with fewer steps outperforms the quality of results sampled by current methods with randomness from the same pre-trained diffusion-based SR model.
  arXiv  Detail & Related papers  (2023-05-24T17:09:54Z)
• SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models [0.49478969093606673]
  A potent class of generative models known as Diffusion Probabilistic Models (DPMs) has become prominent. Despite being quick, such solvers do not usually reach the optimal quality achieved by available slow SDE solvers. Our goal is to propose SDE solvers that reach optimal quality without requiring several hundreds or thousands of NFEs to achieve that goal.
  arXiv  Detail & Related papers  (2023-05-23T17:19:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.