Iterated Schrödinger bridge approximation to Wasserstein Gradient Flows

• URL: http://arxiv.org/abs/2406.10823v1
• Date: Sun, 16 Jun 2024 07:23:26 GMT
• Title: Iterated Schrödinger bridge approximation to Wasserstein Gradient Flows
• Authors: Medha Agarwal, Zaid Harchaoui, Garrett Mulcahy, Soumik Pal,
• Abstract summary: We introduce a novel discretization scheme for Wasserstein gradient flows that involves successively computing Schr"odinger bridges with the same marginals. The proposed scheme has two advantages: one, it avoids the use of the score function, and, two, it is amenable to particle-based approximations using the Sinkhorn algorithm.
• Score: 1.5561923713703105
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel discretization scheme for Wasserstein gradient flows that involves successively computing Schr\"{o}dinger bridges with the same marginals. This is different from both the forward/geodesic approximation and the backward/Jordan-Kinderlehrer-Otto (JKO) approximations. The proposed scheme has two advantages: one, it avoids the use of the score function, and, two, it is amenable to particle-based approximations using the Sinkhorn algorithm. Our proof hinges upon showing that relative entropy between the Schr\"{o}dinger bridge with the same marginals at temperature $\epsilon$ and the joint distribution of a stationary Langevin diffusion at times zero and $\epsilon$ is of the order $o(\epsilon^2)$ with an explicit dependence given by Fisher information. Owing to this inequality, we can show, using a triangular approximation argument, that the interpolated iterated application of the Schr\"{o}dinger bridge approximation converge to the Wasserstein gradient flow, for a class of gradient flows, including the heat flow. The results also provide a probabilistic and rigorous framework for the convergence of the self-attention mechanisms in transformer networks to the solutions of heat flows, first observed in the inspiring work SABP22 in machine learning research.

Related papers

• Wasserstein Mirror Gradient Flow as the limit of the Sinkhorn Algorithm [0.15749416770494706]
  We prove that the sequence of marginals obtained from the iterations of the Sinkhorn algorithm converges to an absolutely continuous curve on the $2$-Wasserstein space. This limit, which we call the Sinkhorn flow, is an example of a Wasserstein mirror gradient flow.
  arXiv  Detail & Related papers  (2023-07-31T06:11:47Z)
• Variational Gaussian filtering via Wasserstein gradient flows [6.023171219551961]
  We present a novel approach to approximate Gaussian and mixture-of-Gaussians filtering. Our method relies on a variational approximation via a gradient-flow representation.
  arXiv  Detail & Related papers  (2023-03-11T12:22:35Z)
• Min-Max Optimization Made Simple: Approximating the Proximal Point Method via Contraction Maps [77.8999425439444]
  We present a first-order method that admits near-optimal convergence rates for convex/concave min-max problems. Our work is based on the fact that the update rule of the Proximal Point method can be approximated up to accuracy.
  arXiv  Detail & Related papers  (2023-01-10T12:18:47Z)
• Structural aspects of FRG in quantum tunnelling computations [68.8204255655161]
  We probe both the unidimensional quartic harmonic oscillator and the double well potential. Two partial differential equations for the potential V_k(varphi) and the wave function renormalization Z_k(varphi) are studied.
  arXiv  Detail & Related papers  (2022-06-14T15:23:25Z)
• Mean-Square Analysis with An Application to Optimal Dimension Dependence of Langevin Monte Carlo [60.785586069299356]
  This work provides a general framework for the non-asymotic analysis of sampling error in 2-Wasserstein distance. Our theoretical analysis is further validated by numerical experiments.
  arXiv  Detail & Related papers  (2021-09-08T18:00:05Z)
• Deep Generative Learning via Schr\"{o}dinger Bridge [14.138796631423954]
  We learn a generative model via entropy with a Schr"odinger Bridge. We show that the generative model via Schr"odinger Bridge is comparable with state-of-the-art GANs.
  arXiv  Detail & Related papers  (2021-06-19T03:35:42Z)
• Non-asymptotic convergence bounds for Wasserstein approximation using point clouds [0.0]
  We show how to generate discrete data as if sampled from a model probability distribution. We provide explicit upper bounds for the convergence-type algorithm.
  arXiv  Detail & Related papers  (2021-06-15T06:53:08Z)
• Large-Scale Wasserstein Gradient Flows [84.73670288608025]
  We introduce a scalable scheme to approximate Wasserstein gradient flows. Our approach relies on input neural networks (ICNNs) to discretize the JKO steps. As a result, we can sample from the measure at each step of the gradient diffusion and compute its density.
  arXiv  Detail & Related papers  (2021-06-01T19:21:48Z)
• Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equations [0.3553493344868413]
  We study the Wasserstein distance between the in distribution of an ergodic differential equation and the distribution in the strongly log-concave case. This allows us to study in a unified way a number of different approximations proposed in the literature for the overdamped and underdamped Langevin dynamics.
  arXiv  Detail & Related papers  (2021-04-26T07:50:04Z)
• Continuous Regularized Wasserstein Barycenters [51.620781112674024]
  We introduce a new dual formulation for the regularized Wasserstein barycenter problem. We establish strong duality and use the corresponding primal-dual relationship to parametrize the barycenter implicitly using the dual potentials of regularized transport problems.
  arXiv  Detail & Related papers  (2020-08-28T08:28:06Z)
• Debiased Sinkhorn barycenters [110.79706180350507]
  Entropy regularization in optimal transport (OT) has been the driver of many recent interests for Wasserstein metrics and barycenters in machine learning. We show how this bias is tightly linked to the reference measure that defines the entropy regularizer. We propose debiased Wasserstein barycenters that preserve the best of both worlds: fast Sinkhorn-like iterations without entropy smoothing.
  arXiv  Detail & Related papers  (2020-06-03T23:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.