Elucidating the solution space of extended reverse-time SDE for diffusion models

• URL: http://arxiv.org/abs/2309.06169v3
• Date: Thu, 27 Feb 2025 07:11:01 GMT
• Title: Elucidating the solution space of extended reverse-time SDE for diffusion models
• Authors: Qinpeng Cui, Xinyi Zhang, Qiqi Bao, Qingmin Liao,
• Abstract summary: We formulate the sampling process as an Extended Reverse-Time SDE (ER SDE)<n>We offer exact solutions and approximate solutions for SDE and VE SDE, respectively.<n>We devise efficient high-quality samplers, namely ER-SDE-rs.
• Score: 23.637881476921596
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sampling from Diffusion Models can alternatively be seen as solving differential equations, where there is a challenge in balancing speed and image visual quality. ODE-based samplers offer rapid sampling time but reach a performance limit, whereas SDE-based samplers achieve superior quality, albeit with longer iterations. In this work, we formulate the sampling process as an Extended Reverse-Time SDE (ER SDE), unifying prior explorations into ODEs and SDEs. Theoretically, leveraging the semi-linear structure of ER SDE solutions, we offer exact solutions and approximate solutions for VP SDE and VE SDE, respectively. Based on the approximate solution space of the ER SDE, referred to as one-step prediction errors, we yield mathematical insights elucidating the rapid sampling capability of ODE solvers and the high-quality sampling ability of SDE solvers. Additionally, we unveil that VP SDE solvers stand on par with their VE SDE counterparts. Based on these findings, leveraging the dual advantages of ODE solvers and SDE solvers, we devise efficient high-quality samplers, namely ER-SDE-Solvers. Experimental results demonstrate that ER-SDE-Solvers achieve state-of-the-art performance across all stochastic samplers while maintaining efficiency of deterministic samplers. Specifically, on the ImageNet $128\times128$ dataset, ER-SDE-Solvers obtain 8.33 FID in only 20 function evaluations. Code is available at \href{https://github.com/QinpengCui/ER-SDE-Solver}{https://github.com/QinpengCui/ER-SDE-Solver}

Related papers

• Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling [11.922538785783587]
  We introduce AdaSDE, a novel single-step SDE solver that aims to unify the efficiency of ODEs with the error resilience of SDEs.<n>Specifically, we introduce a single per-step learnable coefficient, estimated via lightweight distillation, which dynamically regulates the error correction strength to accelerate diffusion sampling.<n>At 5 NFE, AdaSDE achieves FID scores of 4.18 on CIFAR-10, 8.05 on FFHQ and 6.96 on LSUN Bedroom.
  arXiv  Detail & Related papers  (2025-10-27T12:53:48Z)
• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation [57.15474515982337]
  Partial differential equations (PDEs) are fundamental to modeling physical systems.<n>Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive.<n>We introduce CodePDE, the first inference framework for generating PDE solvers using large language models.
  arXiv  Detail & Related papers  (2025-05-13T17:58:08Z)
• Mechanistic PDE Networks for Discovery of Governing Equations [52.492158106791365]
  We present Mechanistic PDE Networks, a model for discovery of partial differential equations from data. The represented PDEs are then solved and decoded for specific tasks. We develop a native, GPU-capable, parallel, sparse, and differentiable multigrid solver specialized for linear partial differential equations.
  arXiv  Detail & Related papers  (2025-02-25T17:21:44Z)
• Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers [55.0876373185983]
  We present the Universal PDE solver (Unisolver) capable of solving a wide scope of PDEs. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components. Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• AdjointDEIS: Efficient Gradients for Diffusion Models [2.0795007613453445]
  We show that continuous adjoint equations for diffusion SDEs actually simplify to a simple ODE. We also demonstrate the effectiveness of AdjointDEIS for guided generation with an adversarial attack in the form of the face morphing problem.
  arXiv  Detail & Related papers  (2024-05-23T19:51:33Z)
• Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior [87.55592645191122]
  Score distillation sampling (SDS) and its variants have greatly boosted the development of text-to-3D generation, but are vulnerable to geometry collapse and poor textures yet. We propose a novel and effective "Consistent3D" method that explores the ODE deterministic sampling prior for text-to-3D generation. Experimental results show the efficacy of our Consistent3D in generating high-fidelity and diverse 3D objects and large-scale scenes.
  arXiv  Detail & Related papers  (2024-01-17T08:32:07Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing [24.76844856196734]
  We present a unified probabilistic formulation for diffusion-based image editing, where a latent variable is edited in a task-specific manner. We provide the SDE counterparts for widely used ODE baselines in various tasks including inpainting and image-to-image translation. We also propose SDE-Drag -- a simple yet effective method built upon the SDE formulation for point-based content dragging.
  arXiv  Detail & Related papers  (2023-11-02T17:23:14Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models [66.67616086310662]
  Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed. We propose SA-of-r, which is an improved efficient Adams method for solving diffusion SDE to generate data with high quality.
  arXiv  Detail & Related papers  (2023-09-10T12:44:54Z)
• Weak-PDE-LEARN: A Weak Form Based Approach to Discovering PDEs From Noisy, Limited Data [0.0]
  We introduce Weak-PDE-LEARN, a discovery algorithm that can identify non-linear PDEs from noisy, limited measurements of their solutions. We demonstrate the efficacy of Weak-PDE-LEARN by learning several benchmark PDEs.
  arXiv  Detail & Related papers  (2023-09-09T06:45:15Z)
• SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models [0.49478969093606673]
  A potent class of generative models known as Diffusion Probabilistic Models (DPMs) has become prominent. Despite being quick, such solvers do not usually reach the optimal quality achieved by available slow SDE solvers. Our goal is to propose SDE solvers that reach optimal quality without requiring several hundreds or thousands of NFEs to achieve that goal.
  arXiv  Detail & Related papers  (2023-05-23T17:19:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.