Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities

• URL: http://arxiv.org/abs/2505.01169v2
• Date: Tue, 06 May 2025 08:22:46 GMT
• Title: Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities
• Abstract summary: We present a new loss function for TTFM distillation called the emphinitial/terminal velocity matching (ITVM) loss.<n>Preliminary experiments show that our loss leads to better few-step generation performance on multiple types of datasets.
• Score: 8.766883138603765
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A flow matching model learns a time-dependent vector field $v_t(x)$ that generates a probability path $\{ p_t \}_{0 \leq t \leq 1}$ that interpolates between a well-known noise distribution ($p_0$) and the data distribution ($p_1$). It can be distilled into a two-timed flow model (TTFM) $\phi_{s,x}(t)$ that can transform a sample belonging to the distribution at an initial time $s$ to another belonging to the distribution at a terminal time $t$ in one function evaluation. We present a new loss function for TTFM distillation called the \emph{initial/terminal velocity matching} (ITVM) loss that extends the Lagrangian Flow Map Distillation (LFMD) loss proposed by Boffi et al. by adding redundant terms to match the initial velocities at time $s$, removing the derivative from the terminal velocity term at time $t$, and using a version of the model under training, stabilized by exponential moving averaging (EMA), to compute the target terminal average velocity. Preliminary experiments show that our loss leads to better few-step generation performance on multiple types of datasets and model architectures over baselines.

Related papers

• Same Flow, Different Paths: Variance Reduction in Flow Matching [67.12978375116599]
  We study the choice of this path from an optimization perspective.<n>Our main finding is that the choice of path $g_t$ can fundamentally change the convergence rate of SGD.
  arXiv  Detail & Related papers  (2026-09-15T15:01:04Z)
• Continuous Adversarial MeanFlow Transfer [30.946652462509068]
  Adversarial refinement is proven effective for few-step quality, but not for the finite-interval average velocities that MeanFlow (MF) models predict.<n>We propose MeanFlow-Transfer, which maps heterogeneous source outputs into a shared velocity representation.<n>We then introduce Continuous Adversarial MeanFlow, a post-training stage that extends adversarial flow models from instantaneous velocities to MF's finite-interval average velocities.
  arXiv  Detail & Related papers  (2026-08-20T01:20:31Z)
• Velocity Scheduled Flow Matching [0.0]
  Flow matching trains a neural network to regress the conditional velocity along a linear interpolant between noise and data.<n>We introduce Velocity Scheduled Flow Matching, which replaces the conditional target.<n>VSFM can be sampled under any admissible profile by integrating its ODE on a non-uniform profile.
  arXiv  Detail & Related papers  (2026-07-13T11:50:06Z)
• Asymptotic-Preserving A Posteriori Analysis of Diffusion and Flow-Matching Samplers [0.6768558752130311]
  Diffusion and flow-matching samplers integrate a learned probability-flow ODE from a large noise scale down to a small terminal floor $_min$, at which the score is stiff and the flow develops a boundary layer.<n>We treat $_min$ as a singular-perturbation parameter and determine which fixed-step samplers are solvable.
  arXiv  Detail & Related papers  (2026-07-05T04:48:33Z)
• Demystifying Pipeline Parallelism: First Theory for PipeDream [53.657104889705856]
  In this paper we introduce Randomized PipeDream (PD) as a stale block-SGD abstraction that yields clean non convergence for a PD-style method.<n>We prove that the delay induced by steady-state PD grows as $S2 - S/2 + O(1)$ for $S$ stages, so the stale-read contribution scales as $(2 S4)$, equivalently $(S4/K)$ in the tuned-rate form.
  arXiv  Detail & Related papers  (2026-06-02T11:14:57Z)
• Posterior Augmented Flow Matching [64.1559809786948]
  Posterior-Augmented Flow Matching (PAFM) is a generalization of flow matching (FM)<n>We show that PAFM improves over FM by up to 3.4 FID50K across different model scales.
  arXiv  Detail & Related papers  (2026-05-01T17:59:59Z)
• Duality Models: An Embarrassingly Simple One-step Generation Paradigm [46.10673908377537]
  "One input, one output" paradigm enforces a partition of the training budget.<n> allocating sufficient samples to the multi-step objective leaves the few-step generation undertrained.<n>We propose Duality Models (DuMo) via a "one input, dual output" paradigm.
  arXiv  Detail & Related papers  (2026-02-04T17:16:22Z)
• Pathway to $O(\sqrt{d})$ Complexity bound under Wasserstein metric of flow-based models [1.724966705006084]
  We provide tools to estimate the error of flow-based generative models under the Wasserstein metric.<n>We show this error can be explicitly controlled by two parts: the Lipschitzness of the push-forward maps of the backward flow which scales independently of the dimension.
  arXiv  Detail & Related papers  (2025-12-07T07:26:39Z)
• Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching [60.37045080890305]
  We provide the first analysis of the sample complexity for flow-matching based generative models.<n>We decompose the velocity field estimation error into neural-network approximation error, statistical error due to the finite sample size, and optimization error due to the finite number of optimization steps for estimating the velocity field.
  arXiv  Detail & Related papers  (2025-12-01T05:14:25Z)
• Flow Matching with Semidiscrete Couplings [29.980130720112726]
  Flow models parameterized as time-dependent velocity fields can generate data from noise by integrating an ODE.<n>These models are often trained using flow matching, i.e. by sampling random pairs of noise and target points.<n>Although promising in theory, the OT flow matching (OT-FM) approach is not widely used in practice.
  arXiv  Detail & Related papers  (2025-09-29T21:22:35Z)
• Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models [5.939858158928473]
  Diffusion probabilistic models generate samples by learning to reverse a noise-injection process that transforms data into noise.<n>Reformulating this reverse process as a deterministic probability flow ordinary differential equation (ODE) enables efficient sampling using high-order solvers.<n>Since the score function is typically approximated by a neural network, analyzing the interaction between its regularity, approximation error, and numerical integration error is key to understanding the overall sampling accuracy.
  arXiv  Detail & Related papers  (2025-06-16T03:09:25Z)
• On the Wasserstein Convergence and Straightness of Rectified Flow [54.580605276017096]
  Rectified Flow (RF) is a generative model that aims to learn straight flow trajectories from noise to data.<n>We provide a theoretical analysis of the Wasserstein distance between the sampling distribution of RF and the target distribution.<n>We present general conditions guaranteeing uniqueness and straightness of 1-RF, which is in line with previous empirical findings.
  arXiv  Detail & Related papers  (2024-10-19T02:36:11Z)
• IT$^3$: Idempotent Test-Time Training [95.78053599609044]
  This paper introduces Idempotent Test-Time Training (IT$3$), a novel approach to addressing the challenge of distribution shift. IT$3$ is based on the universal property of idempotence. We demonstrate the versatility of our approach across various tasks, including corrupted image classification.
  arXiv  Detail & Related papers  (2024-10-05T15:39:51Z)
• Theoretical guarantees in KL for Diffusion Flow Matching [9.618473763561418]
  Flow Matching (FM) aims to bridge in finite time the target distribution $nustar$ with an auxiliary distribution $mu$. We obtain non-asymptotics guarantees for Diffusion Flow Matching (DFM) models using as bridge the conditional distribution associated with the Brownian motion.
  arXiv  Detail & Related papers  (2024-09-12T15:19:00Z)
• Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is hard to beat [49.1574468325115]
  This paper presents explicit non-asymptotic bounds on the forward diffusion error in total variation (TV) We parametrise multi-modal data distributions in terms of the distance $R$ to their furthest modes and consider forward diffusions with additive and multiplicative noise.
  arXiv  Detail & Related papers  (2024-08-25T10:28:31Z)
• A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models [45.60426164657739]
  We develop non-asymptotic convergence theory for a diffusion-based sampler. We prove that $d/varepsilon$ are sufficient to approximate the target distribution to within $varepsilon$ total-variation distance. Our results also characterize how $ell$ score estimation errors affect the quality of the data generation processes.
  arXiv  Detail & Related papers  (2024-08-05T09:02:24Z)
• Nearly $d$-Linear Convergence Bounds for Diffusion Models via Stochastic Localization [40.808942894229325]
  We provide the first convergence bounds which are linear in the data dimension. We show that diffusion models require at most $tilde O(fracd log2(1/delta)varepsilon2)$ steps to approximate an arbitrary distribution.
  arXiv  Detail & Related papers  (2023-08-07T16:01:14Z)
• Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models [49.81937966106691]
  We develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach.
  arXiv  Detail & Related papers  (2023-06-15T16:30:08Z)
• Computing high-dimensional optimal transport by flow neural networks [19.859984725284896]
  This work proposes to compute the dynamic OT between two arbitrary distributions $P$ and $Q$ by optimizing a flow model.<n>Our method learns the dynamic OT by finding an invertible flow that minimizes the transport cost.<n>The effectiveness of the proposed model on high-dimensional data is demonstrated by strong empirical performance on OT baselines, image-to-image translation, and high-dimensional DRE.
  arXiv  Detail & Related papers  (2023-05-19T17:48:21Z)
• Restarted Bayesian Online Change-point Detection for Non-Stationary Markov Decision Processes [12.229154524476405]
  We introduce a variant of the Restarted Bayesian Online Change-Point Detection algorithm (R-BOCPD) We propose an improved version of the UCRL2 algorithm for MDPs with state transition kernel sampled from a multinomial distribution. We show that R-BOCPD-UCRL2 enjoys a favorable regret bound of $Oleft(D O sqrtA T K_T logleft (fracTdelta right) + fracK_Tdeltaminlimits_ell.
  arXiv  Detail & Related papers  (2023-04-01T05:26:41Z)
• FeDXL: Provable Federated Learning for Deep X-Risk Optimization [105.17383135458897]
  We tackle a novel federated learning (FL) problem for optimizing a family of X-risks, to which no existing algorithms are applicable. The challenges for designing an FL algorithm for X-risks lie in the non-decomability of the objective over multiple machines and the interdependency between different machines.
  arXiv  Detail & Related papers  (2022-10-26T00:23:36Z)
• Asynchronous Stochastic Optimization Robust to Arbitrary Delays [54.61797739710608]
  We consider optimization with delayed gradients where, at each time stept$, the algorithm makes an update using a stale computation - d_t$ for arbitrary delay $d_t gradient. Our experiments demonstrate the efficacy and robustness of our algorithm in cases where the delay distribution is skewed or heavy-tailed.
  arXiv  Detail & Related papers  (2021-06-22T15:50:45Z)
• Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction [63.41789556777387]
  Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a Markov decision process (MDP) We show that the number of samples needed to yield an entrywise $varepsilon$-accurate estimate of the Q-function is at most on the order of $frac1mu_min (1-gamma)5varepsilon2+ fract_mixmu_min (1-gamma)$ up to some logarithmic factor.
  arXiv  Detail & Related papers  (2020-06-04T17:51:00Z)
• Gravitational-wave parameter estimation with autoregressive neural network flows [0.0]
  We introduce the use of autoregressive normalizing flows for rapid likelihood-free inference of binary black hole system parameters from gravitational-wave data with deep neural networks. A normalizing flow is an invertible mapping on a sample space that can be used to induce a transformation from a simple probability distribution to a more complex one. We build a more powerful latent variable model by incorporating autoregressive flows within the variational autoencoder framework.
  arXiv  Detail & Related papers  (2020-02-18T15:44:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.