Enhanced Diffusion Sampling via Extrapolation with Multiple ODE Solutions

• URL: http://arxiv.org/abs/2504.01855v1
• Date: Wed, 02 Apr 2025 16:06:23 GMT
• Title: Enhanced Diffusion Sampling via Extrapolation with Multiple ODE Solutions
• Authors: Jinyoung Choi, Junoh Kang, Bohyung Han,
• Abstract summary: Diffusion probabilistic models (DPMs) often suffer from high computational costs due to their iterative sampling process.<n>We propose an enhanced ODE-based sampling method for DPMs inspired by Richardson extrapolation, which reduces numerical error and improves convergence rates.
• Score: 41.989657585518984
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Diffusion probabilistic models (DPMs), while effective in generating high-quality samples, often suffer from high computational costs due to their iterative sampling process. To address this, we propose an enhanced ODE-based sampling method for DPMs inspired by Richardson extrapolation, which reduces numerical error and improves convergence rates. Our method, RX-DPM, leverages multiple ODE solutions at intermediate time steps to extrapolate the denoised prediction in DPMs. This significantly enhances the accuracy of estimations for the final sample while maintaining the number of function evaluations (NFEs). Unlike standard Richardson extrapolation, which assumes uniform discretization of the time grid, we develop a more general formulation tailored to arbitrary time step scheduling, guided by local truncation error derived from a baseline sampling method. The simplicity of our approach facilitates accurate estimation of numerical solutions without significant computational overhead, and allows for seamless and convenient integration into various DPMs and solvers. Additionally, RX-DPM provides explicit error estimates, effectively demonstrating the faster convergence as the leading error term's order increases. Through a series of experiments, we show that the proposed method improves the quality of generated samples without requiring additional sampling iterations.

Related papers

• Neural Flow Samplers with Shortcut Models [19.81513273510523]
  Flow-based samplers generate samples by learning a velocity field that satisfies the continuity equation. While importance sampling provides an approximation, it suffers from high variance.
  arXiv  Detail & Related papers  (2025-02-11T07:55:41Z)
• Optimizing Few-Step Sampler for Diffusion Probabilistic Model [0.0]
  Diffusion Probabilistic Models (DPMs) have demonstrated exceptional capability of generating high-quality and diverse images.<n>But their practical application is hindered by the intensive computational cost during inference.
  arXiv  Detail & Related papers  (2024-12-14T10:47:52Z)
• Accelerating Diffusion Sampling with Optimized Time Steps [69.21208434350567]
  Diffusion probabilistic models (DPMs) have shown remarkable performance in high-resolution image synthesis. Their sampling efficiency is still to be desired due to the typically large number of sampling steps. Recent advancements in high-order numerical ODE solvers for DPMs have enabled the generation of high-quality images with much fewer sampling steps.
  arXiv  Detail & Related papers  (2024-02-27T10:13:30Z)
• Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo [30.4930148381328]
  Diffusion-based Monte Carlo (DMC) is a method to sample from a general target distribution beyond the isoperimetric condition. DMC encountered high gradient complexity, resulting in an exponential dependency on the error tolerance $epsilon$ of the obtained samples. We propose RS-DMC, based on a novel recursion-based score estimation method. Our algorithm is provably much faster than the popular Langevin-based algorithms.
  arXiv  Detail & Related papers  (2024-01-12T02:33:57Z)
• AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models [103.41269503488546]
  Existing customization methods require access to multiple reference examples to align pre-trained diffusion probabilistic models with user-provided concepts. This paper aims to address the challenge of DPM customization when the only available supervision is a differentiable metric defined on the generated contents. We propose a novel method AdjointDPM, which first generates new samples from diffusion models by solving the corresponding probability-flow ODEs. It then uses the adjoint sensitivity method to backpropagate the gradients of the loss to the models' parameters.
  arXiv  Detail & Related papers  (2023-07-20T09:06:21Z)
• On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximation [12.882586878998579]
  A popular approach to sample a diffusion-based generative model is to solve an ordinary differential equation (ODE) We consider accelerating several popular ODE-based sampling processes by optimizing certain coefficients via improved integration approximation (IIA) We show that considerably better FID scores can be achieved by using IIA-EDM, IIA-DDIM, and IIA-DPM-r than the original counterparts.
  arXiv  Detail & Related papers  (2023-04-22T06:06:28Z)
• On Calibrating Diffusion Probabilistic Models [78.75538484265292]
  diffusion probabilistic models (DPMs) have achieved promising results in diverse generative tasks. We propose a simple way for calibrating an arbitrary pretrained DPM, with which the score matching loss can be reduced and the lower bounds of model likelihood can be increased. Our calibration method is performed only once and the resulting models can be used repeatedly for sampling.
  arXiv  Detail & Related papers  (2023-02-21T14:14:40Z)
• Accelerating Diffusion Models via Early Stop of the Diffusion Process [114.48426684994179]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved impressive performance on various generation tasks. In practice DDPMs often need hundreds even thousands of denoising steps to obtain a high-quality sample. We propose a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), for DDPMs.
  arXiv  Detail & Related papers  (2022-05-25T06:40:09Z)
• Pseudo Numerical Methods for Diffusion Models on Manifolds [77.40343577960712]
  Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples. DDPMs require hundreds to thousands of iterations to produce final samples. We propose pseudo numerical methods for diffusion models (PNDMs) PNDMs can generate higher quality synthetic images with only 50 steps compared with 1000-step DDIMs (20x speedup)
  arXiv  Detail & Related papers  (2022-02-20T10:37:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.