On Flow Matching KL Divergence

• URL: http://arxiv.org/abs/2511.05480v1
• Date: Fri, 07 Nov 2025 18:47:46 GMT
• Title: On Flow Matching KL Divergence
• Authors: Maojiang Su, Jerry Yao-Chieh Hu, Sophia Pi, Han Liu,
• Abstract summary: We derive a deterministic, non-asymptotic upper bound on the Kullback-Leibler (KL) divergence of the flow-matching distribution approximation.<n>Our results make the statistical efficiency of flow matching comparable to that of diffusion models under the TV distance.
• Score: 18.018526452560728
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We derive a deterministic, non-asymptotic upper bound on the Kullback-Leibler (KL) divergence of the flow-matching distribution approximation. In particular, if the $L_2$ flow-matching loss is bounded by $\epsilon^2 > 0$, then the KL divergence between the true data distribution and the estimated distribution is bounded by $A_1 \epsilon + A_2 \epsilon^2$. Here, the constants $A_1$ and $A_2$ depend only on the regularities of the data and velocity fields. Consequently, this bound implies statistical convergence rates of Flow Matching Transformers under the Total Variation (TV) distance. We show that, flow matching achieves nearly minimax-optimal efficiency in estimating smooth distributions. Our results make the statistical efficiency of flow matching comparable to that of diffusion models under the TV distance. Numerical studies on synthetic and learned velocities corroborate our theory.

Related papers

• Flow Matching is Adaptive to Manifold Structures [32.55405572762157]
  Flow matching is a simulation-based alternative to diffusion-based generative modeling.<n>We show how flow matching adapts to data geometry and circumvents the curse of dimensionality.
  arXiv  Detail & Related papers  (2026-02-25T23:52:32Z)
• Total Variation Rates for Riemannian Flow Matching [8.235086108564998]
  We develop a nonasymptotic Total Variation analysis for RFM samplers.<n>Our key technical ingredient is a differential inequality governing the evolution of TV between two manifold ODE flows.
  arXiv  Detail & Related papers  (2026-02-05T01:06:53Z)
• Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching [60.37045080890305]
  We provide the first analysis of the sample complexity for flow-matching based generative models.<n>We decompose the velocity field estimation error into neural-network approximation error, statistical error due to the finite sample size, and optimization error due to the finite number of optimization steps for estimating the velocity field.
  arXiv  Detail & Related papers  (2025-12-01T05:14:25Z)
• Distribution estimation via Flow Matching with Lipschitz guarantees [0.0]
  Flow Matching, a promising approach in generative modeling, has recently gained popularity.<n>We study the assumptions that lead to controlling this dependency.<n>We derive a convergence rate for the Wasserstein $1$ distance between the estimated distribution and the target distribution.
  arXiv  Detail & Related papers  (2025-09-02T14:04:11Z)
• On the Wasserstein Convergence and Straightness of Rectified Flow [54.580605276017096]
  Rectified Flow (RF) is a generative model that aims to learn straight flow trajectories from noise to data.<n>We provide a theoretical analysis of the Wasserstein distance between the sampling distribution of RF and the target distribution.<n>We present general conditions guaranteeing uniqueness and straightness of 1-RF, which is in line with previous empirical findings.
  arXiv  Detail & Related papers  (2024-10-19T02:36:11Z)
• Kinetic Interacting Particle Langevin Monte Carlo [0.0]
  This paper introduces and analyses interacting underdamped Langevin algorithms, for statistical inference in latent variable models. We propose a diffusion process that evolves jointly in the space of parameters and latent variables. We provide two explicit discretisations of this diffusion as practical algorithms to estimate parameters of statistical models.
  arXiv  Detail & Related papers  (2024-07-08T09:52:46Z)
• Flow matching achieves almost minimax optimal convergence [50.38891696297888]
  Flow matching (FM) has gained significant attention as a simulation-free generative model. This paper discusses the convergence properties of FM for large sample size under the $p$-Wasserstein distance. We establish that FM can achieve an almost minimax optimal convergence rate for $1 leq p leq 2$, presenting the first theoretical evidence that FM can reach convergence rates comparable to those of diffusion models.
  arXiv  Detail & Related papers  (2024-05-31T14:54:51Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Minimizing $f$-Divergences by Interpolating Velocity Fields [6.8979561422924185]
  Wasserstein Gradient Flow can move particles along a path that minimizes the $f$-divergence between the target and particle distributions. To move particles, we need to calculate the corresponding velocity fields derived from a density ratio function between these two distributions.
  arXiv  Detail & Related papers  (2023-05-24T21:23:58Z)
• Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions [9.953088581242845]
  We provide convergence guarantees with complexity for any data distribution with second-order moment. Our result does not rely on any log-concavity or functional inequality assumption. Our theoretical analysis provides comparison between different discrete approximations and may guide the choice of discretization points in practice.
  arXiv  Detail & Related papers  (2022-11-03T15:51:00Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Minimax Optimal Estimation of KL Divergence for Continuous Distributions [56.29748742084386]
  Esting Kullback-Leibler divergence from identical and independently distributed samples is an important problem in various domains. One simple and effective estimator is based on the k nearest neighbor between these samples.
  arXiv  Detail & Related papers  (2020-02-26T16:37:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.