Robust Target Training for Multi-Source Domain Adaptation

• URL: http://arxiv.org/abs/2210.01676v1
• Date: Tue, 4 Oct 2022 15:20:01 GMT
• Title: Robust Target Training for Multi-Source Domain Adaptation
• Authors: Zhongying Deng, Da Li, Yi-Zhe Song, Tao Xiang
• Abstract summary: We propose a novel Bi-level Optimization based Robust Target Training (BORT$2$) method for MSDA. Our proposed method achieves the state of the art performance on three MSDA benchmarks, including the large-scale DomainNet dataset.
• Score: 110.77704026569499
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given multiple labeled source domains and a single target domain, most existing multi-source domain adaptation (MSDA) models are trained on data from all domains jointly in one step. Such an one-step approach limits their ability to adapt to the target domain. This is because the training set is dominated by the more numerous and labeled source domain data. The source-domain-bias can potentially be alleviated by introducing a second training step, where the model is fine-tuned with the unlabeled target domain data only using pseudo labels as supervision. However, the pseudo labels are inevitably noisy and when used unchecked can negatively impact the model performance. To address this problem, we propose a novel Bi-level Optimization based Robust Target Training (BORT$^2$) method for MSDA. Given any existing fully-trained one-step MSDA model, BORT$^2$ turns it to a labeling function to generate pseudo-labels for the target data and trains a target model using pseudo-labeled target data only. Crucially, the target model is a stochastic CNN which is designed to be intrinsically robust against label noise generated by the labeling function. Such a stochastic CNN models each target instance feature as a Gaussian distribution with an entropy maximization regularizer deployed to measure the label uncertainty, which is further exploited to alleviate the negative impact of noisy pseudo labels. Training the labeling function and the target model poses a nested bi-level optimization problem, for which we formulate an elegant solution based on implicit differentiation. Extensive experiments demonstrate that our proposed method achieves the state of the art performance on three MSDA benchmarks, including the large-scale DomainNet dataset. Our code will be available at \url{https://github.com/Zhongying-Deng/BORT2}

Related papers

• Inter-Domain Mixup for Semi-Supervised Domain Adaptation [108.40945109477886]
  Semi-supervised domain adaptation (SSDA) aims to bridge source and target domain distributions, with a small number of target labels available. Existing SSDA work fails to make full use of label information from both source and target domains for feature alignment across domains. This paper presents a novel SSDA approach, Inter-domain Mixup with Neighborhood Expansion (IDMNE), to tackle this issue.
  arXiv  Detail & Related papers  (2024-01-21T10:20:46Z)
• De-Confusing Pseudo-Labels in Source-Free Domain Adaptation [14.954662088592762]
  Source-free domain adaptation aims to adapt a source-trained model to an unlabeled target domain without access to the source data. We introduce a novel noise-learning approach tailored to address noise distribution in domain adaptation settings.
  arXiv  Detail & Related papers  (2024-01-03T10:07:11Z)
• Target and Task specific Source-Free Domain Adaptive Image Segmentation [73.78898054277538]
  We propose a two-stage approach for source-free domain adaptive image segmentation. We focus on generating target-specific pseudo labels while suppressing high entropy regions. In the second stage, we focus on adapting the network for task-specific representation.
  arXiv  Detail & Related papers  (2022-03-29T17:50:22Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• Adaptive Pseudo-Label Refinement by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation [35.728603077621564]
  Existing Unsupervised Domain Adaptation (UDA) methods presumes source and target domain data to be simultaneously available during training. A pre-trained source model is always considered to be available, even though performing poorly on target due to the well-known domain shift problem. We propose a unified method to tackle adaptive noise filtering and pseudo-label refinement.
  arXiv  Detail & Related papers  (2021-03-29T22:18:34Z)
• A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data [69.091485888121]
  Unsupervised domain adaptation assumes that source and target domain data are freely available and usually trained together to reduce the domain gap. We propose a source data-free domain adaptive object detection (SFOD) framework via modeling it into a problem of learning with noisy labels.
  arXiv  Detail & Related papers  (2020-12-10T01:42:35Z)
• Unsupervised Domain Adaptation for Person Re-Identification through Source-Guided Pseudo-Labeling [2.449909275410288]
  Person Re-Identification (re-ID) aims at retrieving images of the same person taken by different cameras. Unsupervised Domain Adaptation (UDA) is an interesting research direction for this challenge as it avoids a costly annotation of the target data. We introduce a framework which relies on a two-branch architecture optimizing classification and triplet loss based metric learning in source and target domains.
  arXiv  Detail & Related papers  (2020-09-20T14:54:42Z)
• Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification [56.97651712118167]
  Person re-identification (re-ID) aims at identifying the same persons' images across different cameras. domain diversities between different datasets pose an evident challenge for adapting the re-ID model trained on one dataset to another one. We propose an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels.
  arXiv  Detail & Related papers  (2020-01-06T12:42:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.