Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptation

• URL: http://arxiv.org/abs/2304.02064v1
• Date: Tue, 4 Apr 2023 18:32:20 GMT
• Title: Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptation
• Authors: Qi Chen, Mario Marchand
• Abstract summary: We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. We propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency.
• Score: 7.6249291891777915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. Concretely, we study joint distribution alignment for supervised MDA with few target labels and unsupervised MDA with pseudo labels, where the latter is relatively hard and less commonly studied. We further provide algorithm-dependent generalization bounds for these two settings, where the generalization is characterized by the mutual information between the parameters and the data. Then we propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. Finally, the mutual information bounds are extended to this algorithm providing a non-vacuous gradient-norm estimation. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency.

Related papers

• MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation [155.0797148367653]
  Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain and an unlabeled target domain. We propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks.
  arXiv  Detail & Related papers  (2024-08-29T12:15:10Z)
• PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization [24.413415998529754]
  We propose a new benchmark Hybrid Domain Generalization (HDG) and a novel metric $H2$-CV, which construct various splits to assess the robustness of algorithms. Our method outperforms state-of-the-art algorithms on multiple datasets, especially improving the robustness when confronting data scarcity.
  arXiv  Detail & Related papers  (2024-04-13T13:41:13Z)
• Multi-source domain adaptation for regression [2.8648412780872845]
  Multi-source domain adaptation (DA) aims at leveraging information from more than one source domain to make predictions in a target domain. We extend a flexible single-source DA algorithm for classification through outcome-coarsening to enable its application to regression problems. We then augment our single-source DA algorithm for regression with ensemble learning to achieve multi-source DA.
  arXiv  Detail & Related papers  (2023-12-09T04:09:37Z)
• Semi-supervised Domain Adaptation for Semantic Segmentation [3.946367634483361]
  We propose a novel two-step semi-supervised dual-domain adaptation (SSDDA) approach to address both cross- and intra-domain gaps in semantic segmentation. We demonstrate that the proposed approach outperforms state-of-the-art methods on two common synthetic-to-real semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-20T16:13:00Z)
• Your Classifier can Secretly Suffice Multi-Source Domain Adaptation [72.47706604261992]
  Multi-Source Domain Adaptation (MSDA) deals with the transfer of task knowledge from multiple labeled source domains to an unlabeled target domain. We present a different perspective to MSDA wherein deep models are observed to implicitly align the domains under label supervision.
  arXiv  Detail & Related papers  (2021-03-20T12:44:13Z)
• Model-Based Domain Generalization [96.84818110323518]
  We propose a novel approach for the domain generalization problem called Model-Based Domain Generalization. Our algorithms beat the current state-of-the-art methods on the very-recently-proposed WILDS benchmark by up to 20 percentage points.
  arXiv  Detail & Related papers  (2021-02-23T00:59:02Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation [4.1799778475823315]
  We propose a framework to enhance performance by meta-learning the initial conditions of existing DA algorithms. We present variants for both multi-source unsupervised domain adaptation (MSDA), and semi-supervised domain adaptation (SSDA) We achieve state of the art results on several DA benchmarks including the largest scale DomainNet.
  arXiv  Detail & Related papers  (2020-04-09T07:48:22Z)
• Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey [53.656086832255944]
  Multi-source domain adaptation (MDA) is a powerful extension in which the labeled data may be collected from multiple sources. MDA has attracted increasing attention in both academia and industry.
  arXiv  Detail & Related papers  (2020-02-26T08:07:58Z)
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation [58.38749495295393]
  Domain adaptation aims to learn a transferable model to bridge the domain shift between one labeled source domain and another sparsely labeled or unlabeled target domain. Recent multi-source domain adaptation (MDA) methods do not consider the pixel-level alignment between sources and target. We propose a novel MDA framework to address these challenges.
  arXiv  Detail & Related papers  (2020-02-19T21:22:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.