SADA: Stability-guided Adaptive Diffusion Acceleration

• URL: http://arxiv.org/abs/2507.17135v1
• Date: Wed, 23 Jul 2025 02:15:45 GMT
• Title: SADA: Stability-guided Adaptive Diffusion Acceleration
• Authors: Ting Jiang, Yixiao Wang, Hancheng Ye, Zishan Shao, Jingwei Sun, Jingyang Zhang, Zekai Chen, Jianyi Zhang, Yiran Chen, Hai Li,
• Abstract summary: Diffusion models have achieved remarkable success in generative tasks but suffer from high computational costs.<n>Existing training-free acceleration strategies that reduce per-step computation cost, while effectively reducing sampling time, demonstrate low faithfulness.<n>We propose Stability-guided Adaptive Diffusion Acceleration (SADA), a novel paradigm that accelerates sampling of ODE-based generative models.
• Score: 24.250318487331228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have achieved remarkable success in generative tasks but suffer from high computational costs due to their iterative sampling process and quadratic attention costs. Existing training-free acceleration strategies that reduce per-step computation cost, while effectively reducing sampling time, demonstrate low faithfulness compared to the original baseline. We hypothesize that this fidelity gap arises because (a) different prompts correspond to varying denoising trajectory, and (b) such methods do not consider the underlying ODE formulation and its numerical solution. In this paper, we propose Stability-guided Adaptive Diffusion Acceleration (SADA), a novel paradigm that unifies step-wise and token-wise sparsity decisions via a single stability criterion to accelerate sampling of ODE-based generative models (Diffusion and Flow-matching). For (a), SADA adaptively allocates sparsity based on the sampling trajectory. For (b), SADA introduces principled approximation schemes that leverage the precise gradient information from the numerical ODE solver. Comprehensive evaluations on SD-2, SDXL, and Flux using both EDM and DPM++ solvers reveal consistent $\ge 1.8\times$ speedups with minimal fidelity degradation (LPIPS $\leq 0.10$ and FID $\leq 4.5$) compared to unmodified baselines, significantly outperforming prior methods. Moreover, SADA adapts seamlessly to other pipelines and modalities: It accelerates ControlNet without any modifications and speeds up MusicLDM by $1.8\times$ with $\sim 0.01$ spectrogram LPIPS.

Related papers

• Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models [53.087070073434845]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face image quality degradation under a low-latency budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as ours), a novel ODE solver that mitigates truncation errors by incorporating multiple parallel gradient evaluations in each ODE step.
  arXiv  Detail & Related papers  (2025-07-20T03:08:06Z)
• VAMO: Efficient Large-Scale Nonconvex Optimization via Adaptive Zeroth Order Variance Reduction [3.130722489512822]
  VAMO combines FO mini-batch gradients with ZO finite-difference probes under an ZOG-style framework.<n>VAMO outperforms established FO and ZO methods, offering a faster, more flexible option for improved efficiency.
  arXiv  Detail & Related papers  (2025-05-20T05:31:15Z)
• Flow-GRPO: Training Flow Matching Models via Online RL [75.70017261794422]
  We propose Flow-GRPO, the first method integrating online reinforcement learning (RL) into flow matching models.<n>Our approach uses two key strategies: (1) an ODE-to-SDE conversion that transforms a deterministic Ordinary Equation (ODE) into an equivalent Differential Equation (SDE) that matches the original model's marginal distribution at all timesteps; and (2) a Denoising Reduction strategy that reduces training denoising steps while retaining the original inference timestep number.
  arXiv  Detail & Related papers  (2025-05-08T17:58:45Z)
• DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation [68.55191764622525]
  Diffusion models (DPMs) have shown remarkable performance in visual synthesis but are computationally expensive due to the need for multiple evaluations during the sampling. Recent predictor synthesis-or diffusion samplers have significantly reduced the required number of evaluations, but inherently suffer from a misalignment issue. We introduce a new fast DPM sampler called DC-CPRr, which leverages dynamic compensation to mitigate the misalignment.
  arXiv  Detail & Related papers  (2024-09-05T17:59:46Z)
• PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores [4.595421654683656]
  Diffusion Probabilistic Models (DPMs) have shown remarkable potential in image generation.<n>Most existing solutions accelerate the sampling process by proposing fast ODE solvers.<n>We propose PFDiff, a novel training-free and timestep-skipping strategy, which enables existing fast ODE solvers to operate with fewer NFE.
  arXiv  Detail & Related papers  (2024-08-16T16:12:44Z)
• Efficient Sampling for Data-Driven Frequency Stability Constraint via Forward-Mode Automatic Differentiation [5.603382086370097]
  We propose a gradient-based data generation method via forward-mode automatic differentiation. In this method, the original dynamic system is augmented with new states that represent the dynamic of sensitivities of the original states. We demonstrate the superior performance of the proposed sampling algorithm, compared with the unrolling differentiation and finite difference.
  arXiv  Detail & Related papers  (2024-07-21T03:50:11Z)
• Accelerating Convergence of Score-Based Diffusion Models, Provably [44.11766377798812]
  Score-based diffusion models often suffer from low sampling speed due to extensive function evaluations needed during the sampling phase. We design novel training-free algorithms to accelerate popular deterministic (i.e., DDIM) and (i.e., DDPM) samplers. Our theory accommodates $ell$-accurate score estimates, and does not require log-concavity or smoothness on the target distribution.
  arXiv  Detail & Related papers  (2024-03-06T17:02:39Z)
• Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling [73.5602474095954]
  We study the non-asymptotic performance of approximation schemes with delayed updates under Markovian sampling. Our theoretical findings shed light on the finite-time effects of delays for a broad class of algorithms.
  arXiv  Detail & Related papers  (2024-02-19T03:08:02Z)
• A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models [21.305868355976394]
  In this paper, we propose a unified sampling framework (USF) to study the optional strategies for solver. Under this framework, we reveal that taking different solving strategies at different timesteps may help further decrease the truncation error. We demonstrate that $S3$ can find outstanding solver schedules which outperform the state-of-the-art sampling methods.
  arXiv  Detail & Related papers  (2023-12-12T13:19:40Z)
• Towards More Accurate Diffusion Model Acceleration with A Timestep Aligner [84.97253871387028]
  A diffusion model, which is formulated to produce an image using thousands of denoising steps, usually suffers from a slow inference speed. We propose a timestep aligner that helps find a more accurate integral direction for a particular interval at the minimum cost. Experiments show that our plug-in design can be trained efficiently and boost the inference performance of various state-of-the-art acceleration methods.
  arXiv  Detail & Related papers  (2023-10-14T02:19:07Z)
• Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling [36.50606582918392]
  Exponential solvers have gained prominence by demonstrating state-of-the-art performance. Existing high-order EI-based sampling algorithms rely on degenerate EI solvers. We propose refined EI solvers that fulfill all the order conditions.
  arXiv  Detail & Related papers  (2023-08-04T06:30:40Z)
• Preconditioned Score-based Generative Models [45.66744783988319]
  An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation.<n>We propose a novel preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem.<n>PDS preserves the output distribution of the SGM, with no risk of inducing systematical bias to the original sampling process.
  arXiv  Detail & Related papers  (2023-02-13T16:30:53Z)
• An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification [97.28167655721766]
  We propose a novel doubly accelerated gradient descent (ADSGD) method for sparsity regularized loss minimization problems. We first prove that ADSGD can achieve a linear convergence rate and lower overall computational complexity.
  arXiv  Detail & Related papers  (2022-08-11T22:27:22Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.