Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernels

• URL: http://arxiv.org/abs/2301.11624v3
• Date: Thu, 21 Mar 2024 12:34:14 GMT
• Title: Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernels
• Abstract summary: Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals with non-smooth Riesz kernels show a rich structure. We propose to approximate the backward scheme of Jordan, Kinderlehrer and Otto for computing such Wasserstein gradient flows. We provide analytic formulas for Wasserstein schemes starting at a Dirac measure and show their convergence as the time step size tends to zero.
• Score: 1.3654846342364308
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals with non-smooth Riesz kernels show a rich structure as singular measures can become absolutely continuous ones and conversely. In this paper we contribute to the understanding of such flows. We propose to approximate the backward scheme of Jordan, Kinderlehrer and Otto for computing such Wasserstein gradient flows as well as a forward scheme for so-called Wasserstein steepest descent flows by neural networks (NNs). Since we cannot restrict ourselves to absolutely continuous measures, we have to deal with transport plans and velocity plans instead of usual transport maps and velocity fields. Indeed, we approximate the disintegration of both plans by generative NNs which are learned with respect to appropriate loss functions. In order to evaluate the quality of both neural schemes, we benchmark them on the interaction energy. Here we provide analytic formulas for Wasserstein schemes starting at a Dirac measure and show their convergence as the time step size tends to zero. Finally, we illustrate our neural MMD flows by numerical examples.

Related papers

• Generative Path-Finding Method for Wasserstein Gradient Flow [4.372661069805067]
  We propose a generative path finding framework for Wasserstein gradient paths.<n>GenWGP learns a generative flow that transports mass from an initial density to an unknown equilibrium distribution.<n>Experiments on Fokker Planck and aggregation type problems show that GenWGP matches or exceeds high fidelity reference solutions.
  arXiv  Detail & Related papers  (2026-04-13T14:22:20Z)
• Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences [1.7415937218905126]
  We prove an equivalence between the recently proposed Drifting Model and the Wasserstein gradient flow of the forward KL divergence under kernel density estimation (KDE) approximation.<n>We provide a concise identifiability proof, and a theoretically grounded mixed-divergence strategy.
  arXiv  Detail & Related papers  (2026-03-11T09:48:42Z)
• Physics-informed neural particle flow for the Bayesian update step [0.8220217498103312]
  We propose a physics-informed neural particle flow, which is an amortized inference framework.<n>By embedding a governing partial differential equation (PDE) into the loss function, we train a neural network to approximate the transport velocity field.<n>We demonstrate that the neural parameterization acts as an implicit regularizer, mitigating the stiffness inherent to analytic flows.
  arXiv  Detail & Related papers  (2026-02-26T15:10:45Z)
• Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions [0.9940728137241214]
  gradient flows of the Kullback--Leibler divergence evolve a distribution toward a target density known only up to a normalizing constant.<n>We introduce new gradient flows of the KL divergence with a remarkable combination of properties.<n>They admit accurate interacting-particle approximations in high dimensions, and the per-step cost scales linearly in both the number of particles and the dimension.
  arXiv  Detail & Related papers  (2026-02-05T02:38:56Z)
• Fast Estimation of Wasserstein Distances via Regression on Sliced Wasserstein Distances [70.94157767200342]
  We propose a fast estimation method based on regressing Wasserstein distance on sliced Wasserstein distances.<n>We show that accurate models can be learned from a small number of distribution pairs.<n>Our method consistently provides a better approximation of Wasserstein distance than the state-of-the-art Wasserstein embedding model, Wasserstein Wormhole.
  arXiv  Detail & Related papers  (2025-09-24T19:30:53Z)
• Bayesian inference of mean velocity fields and turbulence models from flow MRI [0.0]
  We devise an algorithm that learns the most likely parameters of an effective viscosity model, and estimates their uncertainties, from mean flow data of a turbulent flow. The algorithm successfully reconstructs the mean flow field and learns the most likely turbulence model parameters without overfitting.
  arXiv  Detail & Related papers  (2024-12-15T18:07:36Z)
• A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization [90.87444114491116]
  This paper studies minimax optimization problems defined over infinite-dimensional function classes of overparametricized two-layer neural networks. We address (i) the convergence of the gradient descent-ascent algorithm and (ii) the representation learning of the neural networks. Results show that the feature representation induced by the neural networks is allowed to deviate from the initial one by the magnitude of $O(alpha-1)$, measured in terms of the Wasserstein distance.
  arXiv  Detail & Related papers  (2024-04-18T16:46:08Z)
• Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel [2.199065293049186]
  conditional flows of the maximum mean discrepancy (MMD) with the negative distance kernel for posterior sampling and conditional generative modeling. We approximate the joint distribution of the ground truth and the observations using discrete Wasserstein gradient flows.
  arXiv  Detail & Related papers  (2023-10-04T11:40:02Z)
• Deep Random Vortex Method for Simulation and Inference of Navier-Stokes Equations [69.5454078868963]
  Navier-Stokes equations are significant partial differential equations that describe the motion of fluids such as liquids and air. With the development of AI techniques, several approaches have been designed to integrate deep neural networks in simulating and inferring the fluid dynamics governed by incompressible Navier-Stokes equations. We propose the emphDeep Random Vortex Method (DRVM), which combines the neural network with a random vortex dynamics system equivalent to the Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-06-20T04:58:09Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• Variational Wasserstein gradient flow [9.901677207027806]
  We propose a scalable proximal gradient type algorithm for Wasserstein gradient flow. Our framework covers all the classical Wasserstein gradient flows including the heat equation and the porous medium equation.
  arXiv  Detail & Related papers  (2021-12-04T20:27:31Z)
• Sliced-Wasserstein Gradient Flows [15.048733056992855]
  Minimizing functionals in the space of probability distributions can be done with Wasserstein gradient flows. This work proposes to use gradient flows in the space of probability measures endowed with the sliced-Wasserstein distance.
  arXiv  Detail & Related papers  (2021-10-21T08:34:26Z)
• Large-Scale Wasserstein Gradient Flows [84.73670288608025]
  We introduce a scalable scheme to approximate Wasserstein gradient flows. Our approach relies on input neural networks (ICNNs) to discretize the JKO steps. As a result, we can sample from the measure at each step of the gradient diffusion and compute its density.
  arXiv  Detail & Related papers  (2021-06-01T19:21:48Z)
• Learning High Dimensional Wasserstein Geodesics [55.086626708837635]
  We propose a new formulation and learning strategy for computing the Wasserstein geodesic between two probability distributions in high dimensions. By applying the method of Lagrange multipliers to the dynamic formulation of the optimal transport (OT) problem, we derive a minimax problem whose saddle point is the Wasserstein geodesic. We then parametrize the functions by deep neural networks and design a sample based bidirectional learning algorithm for training.
  arXiv  Detail & Related papers  (2021-02-05T04:25:28Z)
• A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models [93.24030378630175]
  We propose a novel numerical scheme to optimize the gradient flows for learning energy-based models (EBMs) We derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation. Compared with existing schemes, Wasserstein gradient flow is a smoother and near-optimal numerical scheme to approximate real data densities.
  arXiv  Detail & Related papers  (2019-10-31T02:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.