Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences

• URL: http://arxiv.org/abs/2603.10592v1
• Date: Wed, 11 Mar 2026 09:48:42 GMT
• Title: Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
• Abstract summary: We prove an equivalence between the recently proposed Drifting Model and the Wasserstein gradient flow of the forward KL divergence under kernel density estimation (KDE) approximation.<n>We provide a concise identifiability proof, and a theoretically grounded mixed-divergence strategy.
• Score: 1.7415937218905126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We reveal a precise mathematical framework about a new family of generative models which we call Gradient Flow Drifting. With this framework, we prove an equivalence between the recently proposed Drifting Model and the Wasserstein gradient flow of the forward KL divergence under kernel density estimation (KDE) approximation. Specifically, we prove that the drifting field of drifting model (arXiv:2602.04770) equals, up to a bandwidth-squared scaling factor, the difference of KDE log-density gradients $\nabla \log p_{\mathrm{kde}} - \nabla \log q_{\mathrm{kde}}$, which is exactly the particle velocity field of the Wasserstein-2 gradient flow of $KL(q\|p)$ with KDE-approximated densities. Besides that, this broad family of generative models can also include MMD-based generators, which arises as special cases of Wasserstein gradient flows of different divergences under KDE approximation. We provide a concise identifiability proof, and a theoretically grounded mixed-divergence strategy. We combine reverse KL and $χ^2$ divergence gradient flows to simultaneously avoid mode collapse and mode blurring, and extend this method onto Riemannian manifold which loosens the constraints on the kernel function, and makes this method more suitable for the semantic space. Preliminary experiments on synthetic benchmarks validate the framework.

Related papers

• LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction [59.32834240526981]
  Flow-based generative models are typically sampled by solving a deterministic ordinary differential equation (ODE)<n>Existing GRPO methods for flow models replace the Euler-time ODE with aotropic differential equation (SDE) during training.<n>We introduce LC-GRPO, a flow-based GRPO framework with Langevin correction targeting the marginal distribution at the resulting timestep.
  arXiv  Detail & Related papers  (2026-08-06T04:47:57Z)
• Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability [9.11122093402205]
  gradient descent with large step sizes often diverges away from the gradient flow.<n>"Central Flow", proposed in Cohen et al., provides an accurate approximation to the GD dynamics dynamics.<n>We propose Rod Flow, which better captures GD dynamics for simple toy examples.
  arXiv  Detail & Related papers  (2026-02-01T23:10:44Z)
• Fast Estimation of Wasserstein Distances via Regression on Sliced Wasserstein Distances [70.94157767200342]
  We propose a fast estimation method based on regressing Wasserstein distance on sliced Wasserstein distances.<n>We show that accurate models can be learned from a small number of distribution pairs.<n>Our method consistently provides a better approximation of Wasserstein distance than the state-of-the-art Wasserstein embedding model, Wasserstein Wormhole.
  arXiv  Detail & Related papers  (2025-09-24T19:30:53Z)
• Kernel Approximation of Fisher-Rao Gradient Flows [52.154685604660465]
  We present a rigorous investigation of Fisher-Rao and Wasserstein type gradient flows concerning their gradient structures, flow equations, and their kernel approximations. Specifically, we focus on the Fisher-Rao geometry and its various kernel-based approximations, developing a principled theoretical framework.
  arXiv  Detail & Related papers  (2024-10-27T22:52:08Z)
• Differentially Private Gradient Flow based on the Sliced Wasserstein Distance [59.1056830438845]
  We introduce a novel differentially private generative modeling approach based on a gradient flow in the space of probability measures.<n> Experiments show that our proposed model can generate higher-fidelity data at a low privacy budget.
  arXiv  Detail & Related papers  (2023-12-13T15:47:30Z)
• Bridging the Gap Between Variational Inference and Wasserstein Gradient Flows [6.452626686361619]
  We bridge the gap between variational inference and Wasserstein gradient flows. Under certain conditions, the Bures-Wasserstein gradient flow can be recast as the Euclidean gradient flow. We also offer an alternative perspective on the path-derivative gradient, framing it as a distillation procedure to the Wasserstein gradient flow.
  arXiv  Detail & Related papers  (2023-10-31T00:10:19Z)
• Particle-based Variational Inference with Generalized Wasserstein Gradient Flow [32.37056212527921]
  We propose a ParVI framework, called generalized Wasserstein gradient descent (GWG) We show that GWG exhibits strong convergence guarantees. We also provide an adaptive version that automatically chooses Wasserstein metric to accelerate convergence.
  arXiv  Detail & Related papers  (2023-10-25T10:05:42Z)
• Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel [2.199065293049186]
  conditional flows of the maximum mean discrepancy (MMD) with the negative distance kernel for posterior sampling and conditional generative modeling. We approximate the joint distribution of the ground truth and the observations using discrete Wasserstein gradient flows.
  arXiv  Detail & Related papers  (2023-10-04T11:40:02Z)
• Learning Discretized Neural Networks under Ricci Flow [48.47315844022283]
  We study Discretized Neural Networks (DNNs) composed of low-precision weights and activations.<n>DNNs suffer from either infinite or zero gradients due to the non-differentiable discrete function during training.
  arXiv  Detail & Related papers  (2023-02-07T10:51:53Z)
• Large-Scale Wasserstein Gradient Flows [84.73670288608025]
  We introduce a scalable scheme to approximate Wasserstein gradient flows. Our approach relies on input neural networks (ICNNs) to discretize the JKO steps. As a result, we can sample from the measure at each step of the gradient diffusion and compute its density.
  arXiv  Detail & Related papers  (2021-06-01T19:21:48Z)
• On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow [26.725412498545385]
  We show that a parametric kernelized gradient flow mimics the min-max game in gradient regularized $mathrmMMD$ GAN. We then derive an explicit condition which ensures that gradient descent on the space of the generator in regularized $mathrmMMD$ GAN is globally convergent to the target distribution.
  arXiv  Detail & Related papers  (2020-11-04T16:55:00Z)
• A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models [93.24030378630175]
  We propose a novel numerical scheme to optimize the gradient flows for learning energy-based models (EBMs) We derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation. Compared with existing schemes, Wasserstein gradient flow is a smoother and near-optimal numerical scheme to approximate real data densities.
  arXiv  Detail & Related papers  (2019-10-31T02:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.