Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport

• URL: http://arxiv.org/abs/2305.14777v3
• Date: Tue, 6 Feb 2024 09:41:44 GMT
• Title: Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport
• Authors: Jaemoo Choi, Jaewoong Choi, Myungjoo Kang
• Abstract summary: We propose a novel generative model based on the semi-dual formulation of Unbalanced Optimal Transport (UOT) Unlike OT, UOT relaxes the hard constraint on distribution matching. This approach provides better robustness against outliers, stability during training, and faster convergence. Our model outperforms existing OT-based generative models, achieving FID scores of 2.97 on CIFAR-10 and 6.36 on CelebA-HQ-256.
• Score: 9.980822222343921
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimal Transport (OT) problem investigates a transport map that bridges two distributions while minimizing a given cost function. In this regard, OT between tractable prior distribution and data has been utilized for generative modeling tasks. However, OT-based methods are susceptible to outliers and face optimization challenges during training. In this paper, we propose a novel generative model based on the semi-dual formulation of Unbalanced Optimal Transport (UOT). Unlike OT, UOT relaxes the hard constraint on distribution matching. This approach provides better robustness against outliers, stability during training, and faster convergence. We validate these properties empirically through experiments. Moreover, we study the theoretical upper-bound of divergence between distributions in UOT. Our model outperforms existing OT-based generative models, achieving FID scores of 2.97 on CIFAR-10 and 6.36 on CelebA-HQ-256. The code is available at \url{https://github.com/Jae-Moo/UOTM}.

Related papers

• Double-Bounded Optimal Transport for Advanced Clustering and Classification [58.237576976486544]
  We propose Doubly Bounded Optimal Transport (DB-OT), which assumes that the target distribution is restricted within two boundaries instead of a fixed one. We show that our method can achieve good results with our improved inference scheme in the testing stage.
  arXiv  Detail & Related papers  (2024-01-21T07:43:01Z)
• Optimal Transport-Guided Conditional Score-Based Diffusion Models [63.14903268958398]
  Conditional score-based diffusion model (SBDM) is for conditional generation of target data with paired data as condition, and has achieved great success in image translation. To tackle the applications with partially paired or even unpaired dataset, we propose a novel Optimal Transport-guided Conditional Score-based diffusion model (OTCS) in this paper.
  arXiv  Detail & Related papers  (2023-11-02T13:28:44Z)
• Analyzing and Improving Optimal-Transport-based Adversarial Networks [9.980822222343921]
  Optimal Transport (OT) problem aims to find a transport plan that bridges two distributions while minimizing a given cost function. OT theory has been widely utilized in generative modeling. Our approach achieves a FID score of 2.51 on CIFAR-10 and 5.99 on CelebA-HQ-256, outperforming unified OT-based adversarial approaches.
  arXiv  Detail & Related papers  (2023-10-04T06:52:03Z)
• Building the Bridge of Schr\"odinger: A Continuous Entropic Optimal Transport Benchmark [96.06787302688595]
  We propose a novel way to create pairs of probability distributions for which the ground truth OT solution is known by the construction. We use these benchmark pairs to test how well existing neural EOT/SB solvers actually compute the EOT solution.
  arXiv  Detail & Related papers  (2023-06-16T20:03:36Z)
• Turning Normalizing Flows into Monge Maps with Geodesic Gaussian Preserving Flows [0.0]
  This paper introduces a method to transform any trained NF into a more OT-efficient version without changing the final density. We do so by learning a rearrangement of the source (Gaussian) distribution that minimizes the OT cost between the source and the final density. The proposed method leads to smooth flows with reduced OT cost for several existing models without affecting the model performance.
  arXiv  Detail & Related papers  (2022-09-22T09:16:42Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• Convex Smoothed Autoencoder-Optimal Transport model [0.0]
  We develop a new generative model capable of generating samples which resemble the observed data, and is free from mode collapse and mode mixture. Our model is inspired by the recently proposed Autoencoder- Optimal Transport (AE-OT) model and tries to improve on it by addressing the problems faced by the AE-OT model itself.
  arXiv  Detail & Related papers  (2021-01-14T15:55:20Z)
• Comparing Probability Distributions with Conditional Transport [63.11403041984197]
  We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
  arXiv  Detail & Related papers  (2020-12-28T05:14:22Z)
• Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation [120.69747175899421]
  Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. We propose a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions.
  arXiv  Detail & Related papers  (2020-10-12T17:13:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.