Vision Transformer-based Adversarial Domain Adaptation

• URL: http://arxiv.org/abs/2404.15817v1
• Date: Wed, 24 Apr 2024 11:41:28 GMT
• Title: Vision Transformer-based Adversarial Domain Adaptation
• Authors: Yahan Li, Yuan Wu,
• Abstract summary: Vision transformer (ViT) has attracted tremendous attention since its emergence and has been widely used in various computer vision tasks. In this paper, we fill this gap by employing the ViT as the feature extractor in adversarial domain adaptation. We empirically demonstrate that ViT can be a plug-and-play component in adversarial domain adaptation.
• Score: 5.611768906855499
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised domain adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. The most recent UDA methods always resort to adversarial training to yield state-of-the-art results and a dominant number of existing UDA methods employ convolutional neural networks (CNNs) as feature extractors to learn domain invariant features. Vision transformer (ViT) has attracted tremendous attention since its emergence and has been widely used in various computer vision tasks, such as image classification, object detection, and semantic segmentation, yet its potential in adversarial domain adaptation has never been investigated. In this paper, we fill this gap by employing the ViT as the feature extractor in adversarial domain adaptation. Moreover, we empirically demonstrate that ViT can be a plug-and-play component in adversarial domain adaptation, which means directly replacing the CNN-based feature extractor in existing UDA methods with the ViT-based feature extractor can easily obtain performance improvement. The code is available at https://github.com/LluckyYH/VT-ADA.

Related papers

• Transferable-guided Attention Is All You Need for Video Domain Adaptation [42.642008092347986]
  Unsupervised adaptation (UDA) in videos is a challenging task that remains not well explored compared to image-based UDA techniques. Our key idea is to use transformer layers as a feature encoder and incorporate spatial and temporal transferability relationships into the attention mechanism. A Transferable-guided Attention (TransferAttn) framework is then developed to exploit the capacity of the transformer to adapt cross-domain knowledge.
  arXiv  Detail & Related papers  (2024-07-01T15:29:27Z)
• Improving Source-Free Target Adaptation with Vision Transformers Leveraging Domain Representation Images [8.626222763097335]
  Unsupervised Domain Adaptation (UDA) methods facilitate knowledge transfer from a labeled source domain to an unlabeled target domain. This paper presents an innovative method to bolster ViT performance in source-free target adaptation, beginning with an evaluation of how key, query, and value elements affect ViT outcomes. Domain Representation Images (DRIs) act as domain-specific markers, effortlessly merging with the training regimen.
  arXiv  Detail & Related papers  (2023-11-21T13:26:13Z)
• Unsupervised Domain Adaptation for Video Transformers in Action Recognition [76.31442702219461]
  We propose a simple and novel UDA approach for video action recognition. Our approach builds a robust source model that better generalises to target domain. We report results on two video action benchmarks recognition for UDA.
  arXiv  Detail & Related papers  (2022-07-26T12:17:39Z)
• Towards Unsupervised Domain Adaptation via Domain-Transformer [0.0]
  We propose the Domain-Transformer (DoT) for Unsupervised Domain Adaptation (UDA) DoT integrates the CNN-backbones and the core attention mechanism of Transformers from a new perspective. It achieves the local semantic consistency across domains, where the domain-level attention and manifold regularization are explored.
  arXiv  Detail & Related papers  (2022-02-24T02:30:15Z)
• Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement [79.2994130944482]
  We design a Domain Disentanglement Faster-RCNN (DDF) to eliminate the source-specific information in the features for detection task learning. Our DDF method facilitates the feature disentanglement at the global and local stages, with a Global Triplet Disentanglement (GTD) module and an Instance Similarity Disentanglement (ISD) module. By outperforming state-of-the-art methods on four benchmark UDA object detection tasks, our DDF method is demonstrated to be effective with wide applicability.
  arXiv  Detail & Related papers  (2022-01-06T05:43:01Z)
• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation [44.06904757181245]
  Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to a different unlabeled target domain. One fundamental problem for the category level based UDA is the production of pseudo labels for samples in target domain. We design a two-way center-aware labeling algorithm to produce pseudo labels for target samples. Along with the pseudo labels, a weight-sharing triple-branch transformer framework is proposed to apply self-attention and cross-attention for source/target feature learning and source-target domain alignment.
  arXiv  Detail & Related papers  (2021-09-13T17:59:07Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation [54.61786380919243]
  Unsupervised domain adaptation (UDA) aims to transfer the knowledge learnt from a labeled source domain to an unlabeled target domain. Previous work is mainly built upon convolutional neural networks (CNNs) to learn domain-invariant representations. With the recent exponential increase in applying Vision Transformer (ViT) to vision tasks, the capability of ViT in adapting cross-domain knowledge remains unexplored in the literature.
  arXiv  Detail & Related papers  (2021-08-12T22:37:43Z)
• Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers [141.70707071815653]
  We propose a novel Sequence Feature Alignment (SFA) method that is specially designed for the adaptation of detection transformers. SFA consists of a domain query-based feature alignment (DQFA) module and a token-wise feature alignment (TDA) module. Experiments on three challenging benchmarks show that SFA outperforms state-of-the-art domain adaptive object detection methods.
  arXiv  Detail & Related papers  (2021-07-27T07:17:12Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.