When OT meets MoM: Robust estimation of Wasserstein Distance
- URL: http://arxiv.org/abs/2006.10325v3
- Date: Fri, 18 Feb 2022 17:46:46 GMT
- Title: When OT meets MoM: Robust estimation of Wasserstein Distance
- Authors: Guillaume Staerman, Pierre Laforgue, Pavlo Mozharovskyi, Florence
d'Alch\'e-Buc
- Abstract summary: We consider the problem of estimating the Wasserstein distance between two probability distributions when observations are polluted by outliers.
We introduce and discuss novel MoM-based robust estimators whose consistency is studied under a data contamination model.
We propose a simple MoM-based re-weighting scheme that could be used in conjunction with the Sinkhorn algorithm.
- Score: 8.812837829361923
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Issued from Optimal Transport, the Wasserstein distance has gained importance
in Machine Learning due to its appealing geometrical properties and the
increasing availability of efficient approximations. In this work, we consider
the problem of estimating the Wasserstein distance between two probability
distributions when observations are polluted by outliers. To that end, we
investigate how to leverage Medians of Means (MoM) estimators to robustify the
estimation of Wasserstein distance. Exploiting the dual Kantorovitch
formulation of Wasserstein distance, we introduce and discuss novel MoM-based
robust estimators whose consistency is studied under a data contamination model
and for which convergence rates are provided. These MoM estimators enable to
make Wasserstein Generative Adversarial Network (WGAN) robust to outliers, as
witnessed by an empirical study on two benchmarks CIFAR10 and Fashion MNIST.
Eventually, we discuss how to combine MoM with the entropy-regularized
approximation of the Wasserstein distance and propose a simple MoM-based
re-weighting scheme that could be used in conjunction with the Sinkhorn
algorithm.
Related papers
- Conditional Wasserstein Distances with Applications in Bayesian OT Flow Matching [1.609940380983903]
In inverse problems, many conditional generative models approximate the posterior measure by minimizing a distance between the joint measure and its learned approximation.
We introduce a conditional Wasserstein distance via a set of restricted couplings that equals the expected Wasserstein distance of the posteriors.
We derive theoretical properties of the conditional Wasserstein distance, characterize the corresponding geodesics and velocity fields as well as the flow ODEs.
arXiv Detail & Related papers (2024-03-27T15:54:55Z) - Federated Wasserstein Distance [16.892296712204597]
We introduce a principled way of computing the Wasserstein distance between two distributions in a federated manner.
We show how to estimate the Wasserstein distance between two samples stored and kept on different devices/clients whilst a central entity/server orchestrates the computations.
arXiv Detail & Related papers (2023-10-03T11:30:50Z) - Mutual Wasserstein Discrepancy Minimization for Sequential
Recommendation [82.0801585843835]
We propose a novel self-supervised learning framework based on Mutual WasserStein discrepancy minimization MStein for the sequential recommendation.
We also propose a novel contrastive learning loss based on Wasserstein Discrepancy Measurement.
arXiv Detail & Related papers (2023-01-28T13:38:48Z) - Estimation and inference for the Wasserstein distance between mixing measures in topic models [18.66039789963639]
The Wasserstein distance between mixing measures has come to occupy a central place in the statistical analysis of mixture models.
This work proposes a new canonical interpretation of this distance and provides tools to perform inference on the Wasserstein distance in topic models.
arXiv Detail & Related papers (2022-06-26T02:33:40Z) - Learning High Dimensional Wasserstein Geodesics [55.086626708837635]
We propose a new formulation and learning strategy for computing the Wasserstein geodesic between two probability distributions in high dimensions.
By applying the method of Lagrange multipliers to the dynamic formulation of the optimal transport (OT) problem, we derive a minimax problem whose saddle point is the Wasserstein geodesic.
We then parametrize the functions by deep neural networks and design a sample based bidirectional learning algorithm for training.
arXiv Detail & Related papers (2021-02-05T04:25:28Z) - Continuous Wasserstein-2 Barycenter Estimation without Minimax
Optimization [94.18714844247766]
Wasserstein barycenters provide a geometric notion of the weighted average of probability measures based on optimal transport.
We present a scalable algorithm to compute Wasserstein-2 barycenters given sample access to the input measures.
arXiv Detail & Related papers (2021-02-02T21:01:13Z) - Two-sample Test using Projected Wasserstein Distance [18.46110328123008]
We develop a projected Wasserstein distance for the two-sample test, a fundamental problem in statistics and machine learning.
A key contribution is to couple optimal projection to find the low dimensional linear mapping to maximize the Wasserstein distance between projected probability distributions.
arXiv Detail & Related papers (2020-10-22T18:08:58Z) - Continuous Regularized Wasserstein Barycenters [51.620781112674024]
We introduce a new dual formulation for the regularized Wasserstein barycenter problem.
We establish strong duality and use the corresponding primal-dual relationship to parametrize the barycenter implicitly using the dual potentials of regularized transport problems.
arXiv Detail & Related papers (2020-08-28T08:28:06Z) - On Projection Robust Optimal Transport: Sample Complexity and Model
Misspecification [101.0377583883137]
Projection robust (PR) OT seeks to maximize the OT cost between two measures by choosing a $k$-dimensional subspace onto which they can be projected.
Our first contribution is to establish several fundamental statistical properties of PR Wasserstein distances.
Next, we propose the integral PR Wasserstein (IPRW) distance as an alternative to the PRW distance, by averaging rather than optimizing on subspaces.
arXiv Detail & Related papers (2020-06-22T14:35:33Z) - Projection Robust Wasserstein Distance and Riemannian Optimization [107.93250306339694]
We show that projection robustly solidstein (PRW) is a robust variant of Wasserstein projection (WPP)
This paper provides a first step into the computation of the PRW distance and provides the links between their theory and experiments on and real data.
arXiv Detail & Related papers (2020-06-12T20:40:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.