BoMb-OT: On Batch of Mini-batches Optimal Transport
- URL: http://arxiv.org/abs/2102.05912v1
- Date: Thu, 11 Feb 2021 09:56:25 GMT
- Title: BoMb-OT: On Batch of Mini-batches Optimal Transport
- Authors: Khai Nguyen, Quoc Nguyen, Nhat Ho, Tung Pham, Hung Bui, Dinh Phung,
Trung Le
- Abstract summary: Mini-batch optimal transport (m-OT) has been successfully used in practical applications that involve probability measures with intractable density.
We propose a novel mini-batching scheme for optimal transport, named Batch of Mini-batches Optimal Transport (BoMb-OT)
We show that the new mini-batching scheme can estimate a better transportation plan between two original measures than m-OT.
- Score: 23.602237930502948
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Mini-batch optimal transport (m-OT) has been successfully used in practical
applications that involve probability measures with intractable density, or
probability measures with a very high number of supports. The m-OT solves
several sparser optimal transport problems and then returns the average of
their costs and transportation plans. Despite its scalability advantage, m-OT
is not a proper metric between probability measures since it does not satisfy
the identity property. To address this problem, we propose a novel
mini-batching scheme for optimal transport, named Batch of Mini-batches Optimal
Transport (BoMb-OT), that can be formulated as a well-defined distance on the
space of probability measures. Furthermore, we show that the m-OT is a limit of
the entropic regularized version of the proposed BoMb-OT when the regularized
parameter goes to infinity. We carry out extensive experiments to show that the
new mini-batching scheme can estimate a better transportation plan between two
original measures than m-OT. It leads to a favorable performance of BoMb-OT in
the matching and color transfer tasks. Furthermore, we observe that BoMb-OT
also provides a better objective loss than m-OT for doing approximate Bayesian
computation, estimating parameters of interest in parametric generative models,
and learning non-parametric generative models with gradient flow.
Related papers
- Expected Sliced Transport Plans [9.33181953215826]
We propose a "lifting" operation to extend one-dimensional optimal transport plans back to the original space of the measures.
We prove that using the EST plan to weight the sum of the individual Euclidean costs for moving from one point to another results in a valid metric between the input discrete probability measures.
arXiv Detail & Related papers (2024-10-16T02:44:36Z) - Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
We tackle the task of sampling from a probability density as transporting a tractable density function to the target.
We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions.
PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
arXiv Detail & Related papers (2024-07-10T17:39:50Z) - MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation [80.47072100963017]
We introduce a novel and low-compute algorithm, Model Merging with Amortized Pareto Front (MAP)
MAP efficiently identifies a set of scaling coefficients for merging multiple models, reflecting the trade-offs involved.
We also introduce Bayesian MAP for scenarios with a relatively low number of tasks and Nested MAP for situations with a high number of tasks, further reducing the computational cost of evaluation.
arXiv Detail & Related papers (2024-06-11T17:55:25Z) - Light and Optimal Schrödinger Bridge Matching [67.93806073192938]
We propose a novel procedure to learn Schr"odinger Bridges (SB) which we call the textbf Schr"odinger bridge matching.
We show that the optimal bridge matching objective coincides with the recently discovered energy-based modeling (EBM) objectives to learn EOT/SB.
We develop a light solver (which we call LightSB-M) to implement optimal matching in practice using the mixture parameterization of the Schr"odinger potential.
arXiv Detail & Related papers (2024-02-05T17:17:57Z) - Entropic Neural Optimal Transport via Diffusion Processes [105.34822201378763]
We propose a novel neural algorithm for the fundamental problem of computing the entropic optimal transport (EOT) plan between continuous probability distributions.
Our algorithm is based on the saddle point reformulation of the dynamic version of EOT which is known as the Schr"odinger Bridge problem.
In contrast to the prior methods for large-scale EOT, our algorithm is end-to-end and consists of a single learning step.
arXiv Detail & Related papers (2022-11-02T14:35:13Z) - Low-rank Optimal Transport: Approximation, Statistics and Debiasing [51.50788603386766]
Low-rank optimal transport (LOT) approach advocated in citescetbon 2021lowrank
LOT is seen as a legitimate contender to entropic regularization when compared on properties of interest.
We target each of these areas in this paper in order to cement the impact of low-rank approaches in computational OT.
arXiv Detail & Related papers (2022-05-24T20:51:37Z) - An Efficient Mini-batch Method via Partial Transportation [10.127116789814488]
Mini-batch optimal transport (m-OT) has been widely used to deal with the memory issue of OT in large-scale applications.
We propose a novel mini-batch method by using partial optimal transport (POT) between mini-batch empirical measures.
We show that m-POT is better than m-OT deep domain adaptation applications while having comparable performance with m-UOT.
arXiv Detail & Related papers (2021-08-22T05:45:48Z) - Unbalanced minibatch Optimal Transport; applications to Domain
Adaptation [8.889304968879163]
Optimal transport distances have found many applications in machine learning for their capacity to compare non-parametric probability distributions.
We argue that the same minibatch strategy coupled with unbalanced optimal transport can yield more robust behavior.
Our experimental study shows that in challenging problems associated to domain adaptation, the use of unbalanced optimal transport leads to significantly better results, competing with or surpassing recent baselines.
arXiv Detail & Related papers (2021-03-05T11:15:47Z) - Minibatch optimal transport distances; analysis and applications [9.574645423576932]
Optimal transport distances have become a classic tool to compare probability distributions and have found many applications in machine learning.
A common workaround is to compute these distances on minibatches to average the outcome of several smaller optimal transport problems.
We propose in this paper an extended analysis of this practice, which effects were previously studied in restricted cases.
arXiv Detail & Related papers (2021-01-05T21:29:31Z) - Comparing Probability Distributions with Conditional Transport [63.11403041984197]
We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost.
ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute.
On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
arXiv Detail & Related papers (2020-12-28T05:14:22Z) - Occam's Ghost [0.0]
Minimizing the total bit requirement of a model of a dataset favors smaller derivatives, smoother probability density function estimates and most importantly, a phase space with fewer relevant parameters.
It is also shown, how it can be applied to any smooth, non-parametric probability density estimator.
arXiv Detail & Related papers (2020-06-15T20:25:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.