Analyzing and Improving Optimal-Transport-based Adversarial Networks

• URL: http://arxiv.org/abs/2310.02611v2
• Date: Thu, 7 Mar 2024 05:13:11 GMT
• Title: Analyzing and Improving Optimal-Transport-based Adversarial Networks
• Authors: Jaemoo Choi, Jaewoong Choi, Myungjoo Kang
• Abstract summary: Optimal Transport (OT) problem aims to find a transport plan that bridges two distributions while minimizing a given cost function. OT theory has been widely utilized in generative modeling. Our approach achieves a FID score of 2.51 on CIFAR-10 and 5.99 on CelebA-HQ-256, outperforming unified OT-based adversarial approaches.
• Score: 9.980822222343921
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimal Transport (OT) problem aims to find a transport plan that bridges two distributions while minimizing a given cost function. OT theory has been widely utilized in generative modeling. In the beginning, OT distance has been used as a measure for assessing the distance between data and generated distributions. Recently, OT transport map between data and prior distributions has been utilized as a generative model. These OT-based generative models share a similar adversarial training objective. In this paper, we begin by unifying these OT-based adversarial methods within a single framework. Then, we elucidate the role of each component in training dynamics through a comprehensive analysis of this unified framework. Moreover, we suggest a simple but novel method that improves the previously best-performing OT-based model. Intuitively, our approach conducts a gradual refinement of the generated distribution, progressively aligning it with the data distribution. Our approach achieves a FID score of 2.51 on CIFAR-10 and 5.99 on CelebA-HQ-256, outperforming unified OT-based adversarial approaches.

Related papers

• A Statistical Learning Perspective on Semi-dual Adversarial Neural Optimal Transport Solvers [65.28989155951132]
  In this paper, we establish upper bounds on the generalization error of an approximate OT map recovered by the minimax quadratic OT solver. While our analysis focuses on the quadratic OT, we believe that similar bounds could be derived for more general OT formulations.
  arXiv  Detail & Related papers  (2025-02-03T12:37:20Z)
• ParetoFlow: Guided Flows in Multi-Objective Optimization [12.358524770639136]
  In offline multi-objective optimization (MOO), we leverage an offline dataset of designs their associated labels to simultaneously multiple objectives. Recent iteration mainly employ evolutionary and Bayesian optimization, with limited attention given to the generative capabilities inherent in data. Our method achieves state-of-the-art performance across various tasks.
  arXiv  Detail & Related papers  (2024-12-04T21:14:18Z)
• Double-Bounded Optimal Transport for Advanced Clustering and Classification [58.237576976486544]
  We propose Doubly Bounded Optimal Transport (DB-OT), which assumes that the target distribution is restricted within two boundaries instead of a fixed one. We show that our method can achieve good results with our improved inference scheme in the testing stage.
  arXiv  Detail & Related papers  (2024-01-21T07:43:01Z)
• Unifying Distributionally Robust Optimization via Optimal Transport Theory [13.19058156672392]
  This paper introduces a novel approach that unifies these methods into a single framework based on optimal transport. Our proposed approach makes it possible for optimal adversarial distributions to simultaneously perturb likelihood and outcomes. The paper investigates several duality results and presents tractable reformulations that enhance the practical applicability of this unified framework.
  arXiv  Detail & Related papers  (2023-08-10T08:17:55Z)
• Tackling Computational Heterogeneity in FL: A Few Theoretical Insights [68.8204255655161]
  We introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneous data. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.
  arXiv  Detail & Related papers  (2023-07-12T16:28:21Z)
• Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport [9.980822222343921]
  We propose a novel generative model based on the semi-dual formulation of Unbalanced Optimal Transport (UOT) Unlike OT, UOT relaxes the hard constraint on distribution matching. This approach provides better robustness against outliers, stability during training, and faster convergence. Our model outperforms existing OT-based generative models, achieving FID scores of 2.97 on CIFAR-10 and 6.36 on CelebA-HQ-256.
  arXiv  Detail & Related papers  (2023-05-24T06:31:05Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Learning Optimal Transport Between two Empirical Distributions with Normalizing Flows [12.91637880428221]
  We propose to leverage the flexibility of neural networks to learn an approximate optimal transport map. We show that a particular instance of invertible neural networks, namely the normalizing flows, can be used to approximate the solution of this OT problem.
  arXiv  Detail & Related papers  (2022-07-04T08:08:47Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• Learning to Solve Routing Problems via Distributionally Robust Optimization [14.506553345693536]
  Recent deep models for solving routing problems assume a single distribution of nodes for training, which severely impairs their cross-distribution generalization ability. We exploit group distributionally robust optimization (group DRO) to tackle this issue, where we jointly optimize the weights for different groups of distributions and the parameters for the deep model in an interleaved manner during training. We also design a module based on convolutional neural network, which allows the deep model to learn more informative latent pattern among the nodes.
  arXiv  Detail & Related papers  (2022-02-15T08:06:44Z)
• Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
  Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
  arXiv  Detail & Related papers  (2020-02-14T12:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.