A Quantitative Approximation Framework for Flow Distillation in Diffusion Models

• URL: http://arxiv.org/abs/2606.03820v1
• Date: Tue, 02 Jun 2026 16:00:43 GMT
• Title: A Quantitative Approximation Framework for Flow Distillation in Diffusion Models
• Abstract summary: Local approximation errors can be strongly amplified in low-noise multimodal regimes.<n>We show that deep residual compositions efficiently approximate the long-horizon transport.<n>Experiments support the prediction and reduce end-to-end relative MSE by up to 51.9% with 8 segments compared with uniform grids.
• Score: 9.704954388992892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a quantitative approximation framework for diffusion distillation, viewing few-step sampling as error propagation under compositions of learned flow maps. Focusing on trajectory distillation for the probability-flow ODE, we show that local approximation errors can be strongly amplified in low-noise multimodal regimes, where the underlying dynamics become stiff. In an analytically tractable Gaussian-mixture Ornstein--Uhlenbeck setting, we separate two core difficulties: approximating the time-dependent score field and controlling the dynamical amplification governed by the time-integrated Jacobian bound of the probability-flow ODE. On the approximation side, we prove constructive L^p(p_t) guarantees showing that ReLU--ReQU networks approximate the Gaussian-mixture score uniformly over time, with depth and width scaling polylogarithmically in the target accuracy and explicitly with the mixture geometry. On the stability side, we derive an explicit bound L(t) for the spatial Lipschitz constant of the probability-flow velocity and convert it into a flow map stability estimate governed by \int_s^t L(u)\,du, making late-time amplification in stiff regimes computable. Building on these estimates, we prove that deep residual compositions efficiently approximate the long-horizon transport, with global error controlled by the stability amplification factor, and identify a Lipschitz-mismatch regime in which one-step distillation is structurally unfavorable. The resulting theory yields a stability-balanced non-uniform time grid obtained by uniform partitioning in the cumulative stability coordinate. Experiments support the prediction and reduce end-to-end relative MSE by up to 51.9\% with 8 segments compared with uniform grids.

Related papers

• LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction [59.32834240526981]
  Flow-based generative models are typically sampled by solving a deterministic ordinary differential equation (ODE)<n>Existing GRPO methods for flow models replace the Euler-time ODE with aotropic differential equation (SDE) during training.<n>We introduce LC-GRPO, a flow-based GRPO framework with Langevin correction targeting the marginal distribution at the resulting timestep.
  arXiv  Detail & Related papers  (2026-08-06T04:47:57Z)
• A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate [4.625787677472786]
  We introduce a unified framework for the error analysis of generative models based on the entropy production rate of the forward-reverse diffusion process pair.<n>By analyzing the entropy production at the level of marginal distributions, rather than in path space, our framework yields a sharp convergence rate of $mathcalO(h2)$ for the Euler-Maruyama sampler.
  arXiv  Detail & Related papers  (2026-08-03T15:49:08Z)
• On the Stability of Spherical Hellinger-Kantorovich Flows and Their Implications for Differential Privacy [3.251449696624754]
  We develop a perturbation theory for SHK gradient flows and quantify how potential discrepancies propagate over time.<n>A uniform bound yields dimension-free, pointwise control of the log-likelihood ratio and Rényi divergence.<n>We apply these results to approximate sampling for the exponential mechanism in differential privacy.
  arXiv  Detail & Related papers  (2026-05-22T17:38:20Z)
• Variational Optimality of Föllmer Processes in Generative Diffusions [20.583125441867434]
  We analyze generative diffusions that transport a point mass to a prescribed target distribution over a finite time horizon.<n>We show that the diffusion coefficient can be tuned emphaposteriori without changing the time-marginal distributions.
  arXiv  Detail & Related papers  (2026-02-11T16:15:19Z)
• An Elementary Approach to Scheduling in Generative Diffusion Models [55.171367482496755]
  An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed.<n> Experiments across different datasets and pretrained models demonstrate that the time discretization strategy selected by our approach consistently outperforms baseline and search-based strategies.
  arXiv  Detail & Related papers  (2026-01-20T05:06:26Z)
• Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching [60.37045080890305]
  We provide the first analysis of the sample complexity for flow-matching based generative models.<n>We decompose the velocity field estimation error into neural-network approximation error, statistical error due to the finite sample size, and optimization error due to the finite number of optimization steps for estimating the velocity field.
  arXiv  Detail & Related papers  (2025-12-01T05:14:25Z)
• Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals [3.4240632942024685]
  We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score based MCMC method that is deterministic, resulting in a deterministic evolution for particles.
  arXiv  Detail & Related papers  (2023-08-28T23:51:33Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• Differentiable Annealed Importance Sampling and the Perils of Gradient Noise [68.44523807580438]
  Annealed importance sampling (AIS) and related algorithms are highly effective tools for marginal likelihood estimation. Differentiability is a desirable property as it would admit the possibility of optimizing marginal likelihood as an objective. We propose a differentiable algorithm by abandoning Metropolis-Hastings steps, which further unlocks mini-batch computation.
  arXiv  Detail & Related papers  (2021-07-21T17:10:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.