Cycle Self-Training for Domain Adaptation

• URL: http://arxiv.org/abs/2103.03571v1
• Date: Fri, 5 Mar 2021 10:04:25 GMT
• Title: Cycle Self-Training for Domain Adaptation
• Authors: Hong Liu and Jianmin Wang and Mingsheng Long
• Abstract summary: Cycle Self-Training (CST) is a principled self-training algorithm that enforces pseudo-labels to generalize across domains. CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over prior state-of-the-arts in standard UDA benchmarks.
• Score: 85.14659717421533
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to bridge domain gap. More recently, self-training has been gaining momentum in UDA. Originated from semi-supervised learning, self-training uses unlabeled data efficiently by training on pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from the ground truth labels. Thereby, we propose Cycle Self-Training (CST), a principled self-training algorithm that enforces pseudo-labels to generalize across domains. In the forward step, CST generates target pseudo-labels with a source-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations to make the target classifier perform well on the source data. We introduce the Tsallis entropy, a novel regularization to improve the quality of target pseudo-labels. On quadratic neural networks, we prove that CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over prior state-of-the-arts in standard UDA benchmarks across visual recognition and sentiment analysis tasks.

Related papers

• ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation [4.082799056366928]
  We propose an extensive cut-and-paste strategy (ECAP) to leverage reliable pseudo-labels through data augmentation. ECAP maintains a memory bank of pseudo-labeled target samples throughout training and cut-and-pastes the most confident ones onto the current training batch. We implement ECAP on top of the recent method MIC and boost its performance on two synthetic-to-real domain adaptation benchmarks.
  arXiv  Detail & Related papers  (2024-03-06T17:06:07Z)
• Unsupervised Domain Adaptation for Semantic Segmentation with Pseudo Label Self-Refinement [9.69089112870202]
  We propose an auxiliary pseudo-label refinement network (PRN) for online refining of the pseudo labels and also localizing the pixels whose predicted labels are likely to be noisy. We evaluate our approach on benchmark datasets with three different domain shifts, and our approach consistently performs significantly better than the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-10-25T20:31:07Z)
• GeT: Generative Target Structure Debiasing for Domain Adaptation [67.17025068995835]
  Domain adaptation (DA) aims to transfer knowledge from a fully labeled source to a scarcely labeled or totally unlabeled target under domain shift. Recently, semi-supervised learning-based (SSL) techniques that leverage pseudo labeling have been increasingly used in DA. In this paper, we propose GeT that learns a non-bias target embedding distribution with high quality pseudo labels.
  arXiv  Detail & Related papers  (2023-08-20T08:52:43Z)
• Boosting Cross-Domain Speech Recognition with Self-Supervision [35.01508881708751]
  Cross-domain performance of automatic speech recognition (ASR) could be severely hampered due to mismatch between training and testing distributions. Previous work has shown that self-supervised learning (SSL) or pseudo-labeling (PL) is effective in UDA by exploiting the self-supervisions of unlabeled data. This work presents a systematic UDA framework to fully utilize the unlabeled data with self-supervision in the pre-training and fine-tuning paradigm.
  arXiv  Detail & Related papers  (2022-06-20T14:02:53Z)
• Cycle Label-Consistent Networks for Unsupervised Domain Adaptation [57.29464116557734]
  Domain adaptation aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution. We propose a simple yet efficient domain adaptation method, i.e. Cycle Label-Consistent Network (CLCN), by exploiting the cycle consistency of classification label. We demonstrate the effectiveness of our approach on MNIST-USPS-SVHN, Office-31, Office-Home and Image CLEF-DA benchmarks.
  arXiv  Detail & Related papers  (2022-05-27T13:09:08Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• Adaptive Pseudo-Label Refinement by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation [35.728603077621564]
  Existing Unsupervised Domain Adaptation (UDA) methods presumes source and target domain data to be simultaneously available during training. A pre-trained source model is always considered to be available, even though performing poorly on target due to the well-known domain shift problem. We propose a unified method to tackle adaptive noise filtering and pseudo-label refinement.
  arXiv  Detail & Related papers  (2021-03-29T22:18:34Z)
• Two-phase Pseudo Label Densification for Self-training based Domain Adaptation [93.03265290594278]
  We propose a novel Two-phase Pseudo Label Densification framework, referred to as TPLD. In the first phase, we use sliding window voting to propagate the confident predictions, utilizing intrinsic spatial-correlations in the images. In the second phase, we perform a confidence-based easy-hard classification. To ease the training process and avoid noisy predictions, we introduce the bootstrapping mechanism to the original self-training loss.
  arXiv  Detail & Related papers  (2020-12-09T02:35:25Z)
• Domain Adaptation with Auxiliary Target Domain-Oriented Classifier [115.39091109079622]
  Domain adaptation aims to transfer knowledge from a label-rich but heterogeneous domain to a label-scare domain. One of the most popular SSL techniques is pseudo-labeling that assigns pseudo labels for each unlabeled data. We propose a new pseudo-labeling framework called Auxiliary Target Domain-Oriented (ATDOC) ATDOC alleviates the bias by introducing an auxiliary classifier for target data only, to improve the quality of pseudo labels.
  arXiv  Detail & Related papers  (2020-07-08T15:01:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.