Domain Adaptation and Entanglement: an Optimal Transport Perspective

• URL: http://arxiv.org/abs/2503.08155v1
• Date: Tue, 11 Mar 2025 08:10:03 GMT
• Title: Domain Adaptation and Entanglement: an Optimal Transport Perspective
• Authors: Okan Koç, Alexander Soen, Chao-Kai Chiang, Masashi Sugiyama,
• Abstract summary: Current machine learning systems are brittle in the face of distribution shifts (DS), where the target distribution that the system is tested on differs from the source distribution used to train the system.<n>For deep neural networks, a popular framework for unsupervised domain adaptation (UDA) is domain matching, in which algorithms try to align the marginal distributions in the feature or output space.<n>In this paper, we derive new bounds based on optimal transport that analyze the UDA problem.
• Score: 86.24617989187988
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Current machine learning systems are brittle in the face of distribution shifts (DS), where the target distribution that the system is tested on differs from the source distribution used to train the system. This problem of robustness to DS has been studied extensively in the field of domain adaptation. For deep neural networks, a popular framework for unsupervised domain adaptation (UDA) is domain matching, in which algorithms try to align the marginal distributions in the feature or output space. The current theoretical understanding of these methods, however, is limited and existing theoretical results are not precise enough to characterize their performance in practice. In this paper, we derive new bounds based on optimal transport that analyze the UDA problem. Our new bounds include a term which we dub as \emph{entanglement}, consisting of an expectation of Wasserstein distance between conditionals with respect to changing data distributions. Analysis of the entanglement term provides a novel perspective on the unoptimizable aspects of UDA. In various experiments with multiple models across several DS scenarios, we show that this term can be used to explain the varying performance of UDA algorithms.

Related papers

• Unified Domain Adaptive Semantic Segmentation [96.74199626935294]
  Unsupervised Adaptive Domain Semantic (UDA-SS) aims to transfer the supervision from a labeled source domain to an unlabeled target domain. We propose a Quad-directional Mixup (QuadMix) method, characterized by tackling distinct point attributes and feature inconsistencies. Our method outperforms the state-of-the-art works by large margins on four challenging UDA-SS benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T09:18:49Z)
• Unsupervised Domain Adaptation Based on the Predictive Uncertainty of Models [1.6498361958317636]
  Unsupervised domain adaptation (UDA) aims to improve the prediction performance in the target domain under distribution shifts from the source domain. We present a novel UDA method that learns domain-invariant features that minimize the domain divergence.
  arXiv  Detail & Related papers  (2022-11-16T12:23:32Z)
• Learning Feature Decomposition for Domain Adaptive Monocular Depth Estimation [51.15061013818216]
  Supervised approaches have led to great success with the advance of deep learning, but they rely on large quantities of ground-truth depth annotations. Unsupervised domain adaptation (UDA) transfers knowledge from labeled source data to unlabeled target data, so as to relax the constraint of supervised learning. We propose a novel UDA method for MDE, referred to as Learning Feature Decomposition for Adaptation (LFDA), which learns to decompose the feature space into content and style components.
  arXiv  Detail & Related papers  (2022-07-30T08:05:35Z)
• Generalizing to Unseen Domains with Wasserstein Distributional Robustness under Limited Source Knowledge [22.285156929279207]
  Domain generalization aims at learning a universal model that performs well on unseen target domains. We propose a novel domain generalization framework called Wasserstein Distributionally Robust Domain Generalization (WDRDG)
  arXiv  Detail & Related papers  (2022-07-11T14:46:50Z)
• SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG [25.642435946325925]
  Current EEG technology does not generalize well across domains without expensive supervised re-calibration. We propose a new building block for geometric deep learning, which we denote SPD domain-specific momentum batch normalization (SPDDSMBN) A SPDDSMBN layer can transform domain-specific SPD inputs into domain-invariant SPD outputs, and can be readily applied to multi-source/-target and online UDA scenarios.
  arXiv  Detail & Related papers  (2022-06-02T22:31:36Z)
• Deep Optimal Transport for Domain Adaptation on SPD Manifolds [9.552869120136005]
  neuroimaging data possess the mathematical properties of symmetry and positive definiteness. Applying conventional domain adaptation methods is challenging because these mathematical properties can be disrupted. We introduce a novel geometric deep learning-based approach to manage discrepancies in both marginal and conditional distributions.
  arXiv  Detail & Related papers  (2022-01-15T03:13:02Z)
• Long-tailed Distribution Adaptation [47.21518849423836]
  We formulate Long-tailed recognition as Domain Adaption (LDA), by modeling the long-tailed distribution as an unbalanced domain and the general distribution as a balanced domain. We propose to jointly optimize empirical risks of the unbalanced and balanced domains and approximate their domain divergence by intra-class and inter-class distances. Experiments on benchmark datasets for image recognition, object detection, and instance segmentation validate that our LDA approach achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-10-06T12:15:22Z)
• Improving Transferability of Domain Adaptation Networks Through Domain Alignment Layers [1.3766148734487902]
  Multi-source unsupervised domain adaptation (MSDA) aims at learning a predictor for an unlabeled domain by assigning weak knowledge from a bag of source models. We propose to embed Multi-Source version of DomaIn Alignment Layers (MS-DIAL) at different levels of the predictor. Our approach can improve state-of-the-art MSDA methods, yielding relative gains of up to +30.64% on their classification accuracies.
  arXiv  Detail & Related papers  (2021-09-06T18:41:19Z)
• Heuristic Domain Adaptation [105.59792285047536]
  Heuristic Domain Adaptation Network (HDAN) explicitly learns the domain-invariant and domain-specific representations. Heuristic Domain Adaptation Network (HDAN) has exceeded state-of-the-art on unsupervised DA, multi-source DA and semi-supervised DA.
  arXiv  Detail & Related papers  (2020-11-30T04:21:35Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Few-shot Domain Adaptation by Causal Mechanism Transfer [107.08605582020866]
  We study few-shot supervised domain adaptation (DA) for regression problems, where only a few labeled target domain data and many labeled source domain data are available. Many of the current DA methods base their transfer assumptions on either parametrized distribution shift or apparent distribution similarities. We propose mechanism transfer, a meta-distributional scenario in which a data generating mechanism is invariant among domains.
  arXiv  Detail & Related papers  (2020-02-10T02:16:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.