CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
- URL: http://arxiv.org/abs/2109.06165v1
- Date: Mon, 13 Sep 2021 17:59:07 GMT
- Title: CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
- Authors: Tongkun Xu, Weihua Chen, Pichao Wang, Fan Wang, Hao Li, Rong Jin
- Abstract summary: Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to a different unlabeled target domain.
One fundamental problem for the category level based UDA is the production of pseudo labels for samples in target domain.
We design a two-way center-aware labeling algorithm to produce pseudo labels for target samples.
Along with the pseudo labels, a weight-sharing triple-branch transformer framework is proposed to apply self-attention and cross-attention for source/target feature learning and source-target domain alignment.
- Score: 44.06904757181245
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from
a labeled source domain to a different unlabeled target domain. Most existing
UDA methods focus on learning domain-invariant feature representation, either
from the domain level or category level, using convolution neural networks
(CNNs)-based frameworks. One fundamental problem for the category level based
UDA is the production of pseudo labels for samples in target domain, which are
usually too noisy for accurate domain alignment, inevitably compromising the
UDA performance. With the success of Transformer in various tasks, we find that
the cross-attention in Transformer is robust to the noisy input pairs for
better feature alignment, thus in this paper Transformer is adopted for the
challenging UDA task. Specifically, to generate accurate input pairs, we design
a two-way center-aware labeling algorithm to produce pseudo labels for target
samples. Along with the pseudo labels, a weight-sharing triple-branch
transformer framework is proposed to apply self-attention and cross-attention
for source/target feature learning and source-target domain alignment,
respectively. Such design explicitly enforces the framework to learn
discriminative domain-specific and domain-invariant representations
simultaneously. The proposed method is dubbed CDTrans (cross-domain
transformer), and it provides one of the first attempts to solve UDA tasks with
a pure transformer solution. Extensive experiments show that our proposed
method achieves the best performance on Office-Home, VisDA-2017, and DomainNet
datasets.
Related papers
- QuadFormer: Quadruple Transformer for Unsupervised Domain Adaptation in
Power Line Segmentation of Aerial Images [12.840195641761323]
We propose a novel framework designed for domain adaptive semantic segmentation.
The hierarchical quadruple transformer combines cross-attention and self-attention mechanisms to adapt transferable context.
We present two datasets - ARPLSyn and ARPLReal - to further advance research in unsupervised domain adaptive powerline segmentation.
arXiv Detail & Related papers (2022-11-29T03:15:27Z) - CA-UDA: Class-Aware Unsupervised Domain Adaptation with Optimal
Assignment and Pseudo-Label Refinement [84.10513481953583]
unsupervised domain adaptation (UDA) focuses on the selection of good pseudo-labels as surrogates for the missing labels in the target data.
source domain bias that deteriorates the pseudo-labels can still exist since the shared network of the source and target domains are typically used for the pseudo-label selections.
We propose CA-UDA to improve the quality of the pseudo-labels and UDA results with optimal assignment, a pseudo-label refinement strategy and class-aware domain alignment.
arXiv Detail & Related papers (2022-05-26T18:45:04Z) - Cross-Domain Object Detection with Mean-Teacher Transformer [43.486392965014105]
We propose an end-to-end cross-domain detection transformer based on the mean teacher knowledge transfer (MTKT)
We design three levels of source-target feature alignment strategies based on the architecture of the Transformer, including domain query-based feature alignment (DQFA), bi-level-graph-based prototype alignment (BGPA) and token-wise image feature alignment (TIFA)
Our proposed method achieves state-of-the-art performance on three domain adaptation scenarios, especially the result of Sim10k to Cityscapes scenario is remarkably improved from 52.6 mAP to 57.9 mAP.
arXiv Detail & Related papers (2022-05-03T17:11:55Z) - Exploiting Both Domain-specific and Invariant Knowledge via a Win-win
Transformer for Unsupervised Domain Adaptation [14.623272346517794]
Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain.
Most existing UDA approaches enable knowledge transfer via learning domain-invariant representation and sharing one classifier across two domains.
We propose a Win-Win TRansformer framework (WinTR) that separately explores the domain-specific knowledge for each domain and interchanges cross-domain knowledge.
arXiv Detail & Related papers (2021-11-25T06:45:07Z) - Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training
for Road Segmentation of Remote Sensing Images [93.50240389540252]
Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials.
We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field.
Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
arXiv Detail & Related papers (2021-08-28T09:29:14Z) - Exploring Sequence Feature Alignment for Domain Adaptive Detection
Transformers [141.70707071815653]
We propose a novel Sequence Feature Alignment (SFA) method that is specially designed for the adaptation of detection transformers.
SFA consists of a domain query-based feature alignment (DQFA) module and a token-wise feature alignment (TDA) module.
Experiments on three challenging benchmarks show that SFA outperforms state-of-the-art domain adaptive object detection methods.
arXiv Detail & Related papers (2021-07-27T07:17:12Z) - Cross-domain Contrastive Learning for Unsupervised Domain Adaptation [108.63914324182984]
Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain.
We build upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets.
arXiv Detail & Related papers (2021-06-10T06:32:30Z) - Instance Level Affinity-Based Transfer for Unsupervised Domain
Adaptation [74.71931918541748]
We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA.
We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process.
We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
arXiv Detail & Related papers (2021-04-03T01:33:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.