Improving Source-Free Target Adaptation with Vision Transformers Leveraging Domain Representation Images

• URL: http://arxiv.org/abs/2311.12589v2
• Date: Sat, 2 Dec 2023 11:01:26 GMT
• Title: Improving Source-Free Target Adaptation with Vision Transformers Leveraging Domain Representation Images
• Authors: Gauransh Sawhney, Daksh Dave, Adeel Ahmed, Jiechao Gao, Khalid Saleem
• Abstract summary: Unsupervised Domain Adaptation (UDA) methods facilitate knowledge transfer from a labeled source domain to an unlabeled target domain. This paper presents an innovative method to bolster ViT performance in source-free target adaptation, beginning with an evaluation of how key, query, and value elements affect ViT outcomes. Domain Representation Images (DRIs) act as domain-specific markers, effortlessly merging with the training regimen.
• Score: 8.626222763097335
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Unsupervised Domain Adaptation (UDA) methods facilitate knowledge transfer from a labeled source domain to an unlabeled target domain, navigating the obstacle of domain shift. While Convolutional Neural Networks (CNNs) are a staple in UDA, the rise of Vision Transformers (ViTs) provides new avenues for domain generalization. This paper presents an innovative method to bolster ViT performance in source-free target adaptation, beginning with an evaluation of how key, query, and value elements affect ViT outcomes. Experiments indicate that altering the key component has negligible effects on Transformer performance. Leveraging this discovery, we introduce Domain Representation Images (DRIs), feeding embeddings through the key element. DRIs act as domain-specific markers, effortlessly merging with the training regimen. To assess our method, we perform target adaptation tests on the Cross Instance DRI source-only (SO) control. We measure the efficacy of target adaptation with and without DRIs, against existing benchmarks like SHOT-B* and adaptations via CDTrans. Findings demonstrate that excluding DRIs offers limited gains over SHOT-B*, while their inclusion in the key segment boosts average precision promoting superior domain generalization. This research underscores the vital role of DRIs in enhancing ViT efficiency in UDA scenarios, setting a precedent for further domain adaptation explorations.

Related papers

• Vision Transformer-based Adversarial Domain Adaptation [5.611768906855499]
  Vision transformer (ViT) has attracted tremendous attention since its emergence and has been widely used in various computer vision tasks. In this paper, we fill this gap by employing the ViT as the feature extractor in adversarial domain adaptation. We empirically demonstrate that ViT can be a plug-and-play component in adversarial domain adaptation.
  arXiv  Detail & Related papers  (2024-04-24T11:41:28Z)
• Enhancing Visual Domain Adaptation with Source Preparation [5.287588907230967]
  Domain Adaptation techniques fail to consider the characteristics of the source domain itself. We propose Source Preparation (SP), a method to mitigate source domain biases. We show that SP enhances UDA across a range of visual domains, with improvements up to 40.64% in mIoU over baseline.
  arXiv  Detail & Related papers  (2023-06-16T18:56:44Z)
• Domain-Agnostic Prior for Transfer Semantic Segmentation [197.9378107222422]
  Unsupervised domain adaptation (UDA) is an important topic in the computer vision community. We present a mechanism that regularizes cross-domain representation learning with a domain-agnostic prior (DAP) Our research reveals that UDA benefits much from better proxies, possibly from other data modalities.
  arXiv  Detail & Related papers  (2022-04-06T09:13:25Z)
• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation [44.06904757181245]
  Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to a different unlabeled target domain. One fundamental problem for the category level based UDA is the production of pseudo labels for samples in target domain. We design a two-way center-aware labeling algorithm to produce pseudo labels for target samples. Along with the pseudo labels, a weight-sharing triple-branch transformer framework is proposed to apply self-attention and cross-attention for source/target feature learning and source-target domain alignment.
  arXiv  Detail & Related papers  (2021-09-13T17:59:07Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation [54.61786380919243]
  Unsupervised domain adaptation (UDA) aims to transfer the knowledge learnt from a labeled source domain to an unlabeled target domain. Previous work is mainly built upon convolutional neural networks (CNNs) to learn domain-invariant representations. With the recent exponential increase in applying Vision Transformer (ViT) to vision tasks, the capability of ViT in adapting cross-domain knowledge remains unexplored in the literature.
  arXiv  Detail & Related papers  (2021-08-12T22:37:43Z)
• Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment [32.77436219094282]
  SSDAS employs a few labeled target samples as anchors for adaptive and progressive feature alignment between labeled source samples and unlabeled target samples. In addition, we replace the dissimilar source features by high-confidence target features continuously during the iterative training process. Extensive experiments show the proposed SSDAS greatly outperforms a number of baselines.
  arXiv  Detail & Related papers  (2021-06-05T09:12:50Z)
• Transformer-Based Source-Free Domain Adaptation [134.67078085569017]
  We study the task of source-free domain adaptation (SFDA), where the source data are not available during target adaptation. We propose a generic and effective framework based on Transformer, named TransDA, for learning a generalized model for SFDA.
  arXiv  Detail & Related papers  (2021-05-28T23:06:26Z)
• Disentanglement-based Cross-Domain Feature Augmentation for Effective Unsupervised Domain Adaptive Person Re-identification [87.72851934197936]
  Unsupervised domain adaptive (UDA) person re-identification (ReID) aims to transfer the knowledge from the labeled source domain to the unlabeled target domain for person matching. One challenge is how to generate target domain samples with reliable labels for training. We propose a Disentanglement-based Cross-Domain Feature Augmentation strategy.
  arXiv  Detail & Related papers  (2021-03-25T15:28:41Z)
• Effective Label Propagation for Discriminative Semi-Supervised Domain Adaptation [76.41664929948607]
  Semi-supervised domain adaptation (SSDA) methods have demonstrated great potential in large-scale image classification tasks. We present a novel and effective method to tackle this problem by using effective inter-domain and intra-domain semantic information propagation. Our source code and pre-trained models will be released soon.
  arXiv  Detail & Related papers  (2020-12-04T14:28:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.