Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernels

• URL: http://arxiv.org/abs/2301.11624v3
• Date: Thu, 21 Mar 2024 12:34:14 GMT
• Title: Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernels
• Authors: Fabian Altekrüger, Johannes Hertrich, Gabriele Steidl,
• Abstract summary: Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals with non-smooth Riesz kernels show a rich structure. We propose to approximate the backward scheme of Jordan, Kinderlehrer and Otto for computing such Wasserstein gradient flows. We provide analytic formulas for Wasserstein schemes starting at a Dirac measure and show their convergence as the time step size tends to zero.
• Score: 1.3654846342364308
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals with non-smooth Riesz kernels show a rich structure as singular measures can become absolutely continuous ones and conversely. In this paper we contribute to the understanding of such flows. We propose to approximate the backward scheme of Jordan, Kinderlehrer and Otto for computing such Wasserstein gradient flows as well as a forward scheme for so-called Wasserstein steepest descent flows by neural networks (NNs). Since we cannot restrict ourselves to absolutely continuous measures, we have to deal with transport plans and velocity plans instead of usual transport maps and velocity fields. Indeed, we approximate the disintegration of both plans by generative NNs which are learned with respect to appropriate loss functions. In order to evaluate the quality of both neural schemes, we benchmark them on the interaction energy. Here we provide analytic formulas for Wasserstein schemes starting at a Dirac measure and show their convergence as the time step size tends to zero. Finally, we illustrate our neural MMD flows by numerical examples.

Related papers

• A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization [90.87444114491116]
  This paper studies minimax optimization problems defined over infinite-dimensional function classes of overparametricized two-layer neural networks. We address (i) the convergence of the gradient descent-ascent algorithm and (ii) the representation learning of the neural networks. Results show that the feature representation induced by the neural networks is allowed to deviate from the initial one by the magnitude of $O(alpha-1)$, measured in terms of the Wasserstein distance.
  arXiv  Detail & Related papers  (2024-04-18T16:46:08Z)
• Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel [2.199065293049186]
  conditional flows of the maximum mean discrepancy (MMD) with the negative distance kernel for posterior sampling and conditional generative modeling. We approximate the joint distribution of the ground truth and the observations using discrete Wasserstein gradient flows.
  arXiv  Detail & Related papers  (2023-10-04T11:40:02Z)
• Deep Random Vortex Method for Simulation and Inference of Navier-Stokes Equations [69.5454078868963]
  Navier-Stokes equations are significant partial differential equations that describe the motion of fluids such as liquids and air. With the development of AI techniques, several approaches have been designed to integrate deep neural networks in simulating and inferring the fluid dynamics governed by incompressible Navier-Stokes equations. We propose the emphDeep Random Vortex Method (DRVM), which combines the neural network with a random vortex dynamics system equivalent to the Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-06-20T04:58:09Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• Variational Wasserstein gradient flow [9.901677207027806]
  We propose a scalable proximal gradient type algorithm for Wasserstein gradient flow. Our framework covers all the classical Wasserstein gradient flows including the heat equation and the porous medium equation.
  arXiv  Detail & Related papers  (2021-12-04T20:27:31Z)
• Sliced-Wasserstein Gradient Flows [15.048733056992855]
  Minimizing functionals in the space of probability distributions can be done with Wasserstein gradient flows. This work proposes to use gradient flows in the space of probability measures endowed with the sliced-Wasserstein distance.
  arXiv  Detail & Related papers  (2021-10-21T08:34:26Z)
• Large-Scale Wasserstein Gradient Flows [84.73670288608025]
  We introduce a scalable scheme to approximate Wasserstein gradient flows. Our approach relies on input neural networks (ICNNs) to discretize the JKO steps. As a result, we can sample from the measure at each step of the gradient diffusion and compute its density.
  arXiv  Detail & Related papers  (2021-06-01T19:21:48Z)
• Learning High Dimensional Wasserstein Geodesics [55.086626708837635]
  We propose a new formulation and learning strategy for computing the Wasserstein geodesic between two probability distributions in high dimensions. By applying the method of Lagrange multipliers to the dynamic formulation of the optimal transport (OT) problem, we derive a minimax problem whose saddle point is the Wasserstein geodesic. We then parametrize the functions by deep neural networks and design a sample based bidirectional learning algorithm for training.
  arXiv  Detail & Related papers  (2021-02-05T04:25:28Z)
• A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models [93.24030378630175]
  We propose a novel numerical scheme to optimize the gradient flows for learning energy-based models (EBMs) We derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation. Compared with existing schemes, Wasserstein gradient flow is a smoother and near-optimal numerical scheme to approximate real data densities.
  arXiv  Detail & Related papers  (2019-10-31T02:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.