Optimal Transport for Domain Adaptation through Gaussian Mixture Models

• URL: http://arxiv.org/abs/2403.13847v2
• Date: Wed, 22 Jan 2025 12:47:49 GMT
• Title: Optimal Transport for Domain Adaptation through Gaussian Mixture Models
• Authors: Eduardo Fernandes Montesuma, Fred Maurice Ngolè Mboula, Antoine Souloumiac,
• Abstract summary: Machine learning systems operate under the assumption that training and test data are sampled from a fixed probability distribution.<n>In this work, we explore optimal transport between Gaussian Mixture Models (GMMs), which is conveniently written in terms of the components of source and target GMMs.<n>We experiment with 9 benchmarks, with a total of $85$ adaptation tasks, showing that our methods are more efficient than previous shallow domain adaptation methods.
• Score: 7.292229955481438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning systems operate under the assumption that training and test data are sampled from a fixed probability distribution. However, this assumptions is rarely verified in practice, as the conditions upon which data was acquired are likely to change. In this context, the adaptation of the unsupervised domain requires minimal access to the data of the new conditions for learning models robust to changes in the data distribution. Optimal transport is a theoretically grounded tool for analyzing changes in distribution, especially as it allows the mapping between domains. However, these methods are usually computationally expensive as their complexity scales cubically with the number of samples. In this work, we explore optimal transport between Gaussian Mixture Models (GMMs), which is conveniently written in terms of the components of source and target GMMs. We experiment with 9 benchmarks, with a total of $85$ adaptation tasks, showing that our methods are more efficient than previous shallow domain adaptation methods, and they scale well with number of samples $n$ and dimensions $d$.

Related papers

• Informative Data Mining for One-Shot Cross-Domain Semantic Segmentation [84.82153655786183]
  We propose a novel framework called Informative Data Mining (IDM) to enable efficient one-shot domain adaptation for semantic segmentation. IDM provides an uncertainty-based selection criterion to identify the most informative samples, which facilitates quick adaptation and reduces redundant training. Our approach outperforms existing methods and achieves a new state-of-the-art one-shot performance of 56.7%/55.4% on the GTA5/SYNTHIA to Cityscapes adaptation tasks.
  arXiv  Detail & Related papers  (2023-09-25T15:56:01Z)
• Train/Test-Time Adaptation with Retrieval [129.8579208970529]
  We introduce Train/Test-Time Adaptation with Retrieval ($rm T3AR$), a method to adapt models both at train and test time. $rm T3AR$ adapts a given model to the downstream task using refined pseudo-labels and a self-supervised contrastive objective function. Thanks to the retrieval module, our method gives the user or service provider the possibility to improve model adaptation on the downstream task.
  arXiv  Detail & Related papers  (2023-03-25T02:44:57Z)
• Bidirectional Domain Mixup for Domain Adaptive Semantic Segmentation [73.3083304858763]
  This paper systematically studies the impact of mixup under the domain adaptaive semantic segmentation task. In specific, we achieve domain mixup in two-step: cut and paste. We provide extensive ablation experiments to empirically verify our main components of the framework.
  arXiv  Detail & Related papers  (2023-03-17T05:22:44Z)
• Robustness, Evaluation and Adaptation of Machine Learning Models in the Wild [4.304803366354879]
  We study causes of impaired robustness to domain shifts and present algorithms for training domain robust models. A key source of model brittleness is due to domain overfitting, which our new training algorithms suppress and instead encourage domain-general hypotheses.
  arXiv  Detail & Related papers  (2023-03-05T21:41:16Z)
• Fast OT for Latent Domain Adaptation [25.915629674463286]
  We propose an algorithm that uses optimal transport theory with a verifiably efficient and implementable solution to learn the best latent feature representation. This is achieved by minimizing the cost of transporting the samples from the target domain to the distribution of the source domain.
  arXiv  Detail & Related papers  (2022-10-02T10:25:12Z)
• Connecting adversarial attacks and optimal transport for domain adaptation [116.50515978657002]
  In domain adaptation, the goal is to adapt a classifier trained on the source domain samples to the target domain. In our method, we use optimal transport to map target samples to the domain named source fiction. Our main idea is to generate a source fiction by c-cyclically monotone transformation over the target domain.
  arXiv  Detail & Related papers  (2022-05-30T20:45:55Z)
• Few-Shot Adaptation of Pre-Trained Networks for Domain Shift [17.123505029637055]
  Deep networks are prone to performance degradation when there is a domain shift between the source (training) data and target (test) data. Recent test-time adaptation methods update batch normalization layers of pre-trained source models deployed in new target environments with streaming data to mitigate such performance degradation. We propose a framework for few-shot domain adaptation to address the practical challenges of data-efficient adaptation.
  arXiv  Detail & Related papers  (2022-05-30T16:49:59Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift [3.071136270246468]
  This paper proposes a novel supervised domain adaptation based on two steps. First, we search for an optimal class-dependent transformation from the source to the target domain from a few samples. Second, we use embedding similarity techniques to select the corresponding transformation at inference.
  arXiv  Detail & Related papers  (2022-04-07T10:27:14Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Simpler Certified Radius Maximization by Propagating Covariances [39.851641822878996]
  We show an algorithm for maximizing the certified radius on datasets including Cifar-10, ImageNet, and Places365. We show how satisfying these criteria yields an algorithm for maximizing the certified radius on datasets with moderate depth, with a small compromise in overall accuracy.
  arXiv  Detail & Related papers  (2021-04-13T01:38:36Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation [120.69747175899421]
  Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. We propose a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions.
  arXiv  Detail & Related papers  (2020-10-12T17:13:40Z)
• Open Set Domain Adaptation using Optimal Transport [8.076841611508486]
  We present a 2-step optimal transport approach that performs a mapping from a source distribution to a target distribution. The first step aims at rejecting the samples issued from these new classes using an optimal transport plan. The second step solves the target (class ratio) shift still as an optimal transport problem.
  arXiv  Detail & Related papers  (2020-10-02T15:20:05Z)
• Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
  A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts. We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
  arXiv  Detail & Related papers  (2020-07-06T17:59:30Z)
• Supervised Domain Adaptation using Graph Embedding [86.3361797111839]
  Domain adaptation methods assume that distributions between the two domains are shifted and attempt to realign them. We propose a generic framework based on graph embedding. We show that the proposed approach leads to a powerful Domain Adaptation framework.
  arXiv  Detail & Related papers  (2020-03-09T12:25:13Z)
• Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits [101.68525259222164]
  We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between domains based on sample estimates. We develop a DistanceNet model which uses these distance measures as an additional loss function to be minimized jointly with the task's loss function. We extend this model to a novel DistanceNet-Bandit model, which employs a multi-armed bandit controller to dynamically switch between multiple source domains.
  arXiv  Detail & Related papers  (2020-01-13T15:53:41Z)
• Domain Adaptive Bootstrap Aggregating [5.444459446244819]
  bootstrap aggregating, or bagging, is a popular method for improving stability of predictive algorithms. This article proposes a domain adaptive bagging method coupled with a new iterative nearest neighbor sampler.
  arXiv  Detail & Related papers  (2020-01-12T20:02:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.