Key Design Choices for Double-Transfer in Source-Free Unsupervised Domain Adaptation

• URL: http://arxiv.org/abs/2302.05379v1
• Date: Fri, 10 Feb 2023 17:00:37 GMT
• Title: Key Design Choices for Double-Transfer in Source-Free Unsupervised Domain Adaptation
• Authors: Andrea Maracani, Raffaello Camoriano, Elisa Maiettini, Davide Talon, Lorenzo Rosasco and Lorenzo Natale
• Abstract summary: This paper provides the first in-depth analysis of the main design choices in Source-Free Unsupervised Domain Adaptation (SF-UDA) We pinpoint the normalization approach, pre-training strategy, and backbone architecture as the most critical factors. We show that SF-UDA is competitive also beyond standard benchmarks and backbone architectures, performing on par with UDA at a fraction of the data and computational cost.
• Score: 18.21955526087808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fine-tuning and Domain Adaptation emerged as effective strategies for efficiently transferring deep learning models to new target tasks. However, target domain labels are not accessible in many real-world scenarios. This led to the development of Unsupervised Domain Adaptation (UDA) methods, which only employ unlabeled target samples. Furthermore, efficiency and privacy requirements may also prevent the use of source domain data during the adaptation stage. This challenging setting, known as Source-Free Unsupervised Domain Adaptation (SF-UDA), is gaining interest among researchers and practitioners due to its potential for real-world applications. In this paper, we provide the first in-depth analysis of the main design choices in SF-UDA through a large-scale empirical study across 500 models and 74 domain pairs. We pinpoint the normalization approach, pre-training strategy, and backbone architecture as the most critical factors. Based on our quantitative findings, we propose recipes to best tackle SF-UDA scenarios. Moreover, we show that SF-UDA is competitive also beyond standard benchmarks and backbone architectures, performing on par with UDA at a fraction of the data and computational cost. In the interest of reproducibility, we include the full experimental results and code as supplementary material.

Related papers

• Towards Source-free Domain Adaptive Semantic Segmentation via Importance-aware and Prototype-contrast Learning [26.544837987747766]
  We propose an end-to-end source-free domain adaptation semantic segmentation method via Importance-Aware and Prototype-Contrast learning. The proposed IAPC framework effectively extracts domain-invariant knowledge from the well-trained source model and learns domain-specific knowledge from the unlabeled target domain.
  arXiv  Detail & Related papers  (2023-06-02T15:09:19Z)
• A Comprehensive Survey on Source-free Domain Adaptation [69.17622123344327]
  The research of Source-Free Domain Adaptation (SFDA) has drawn growing attention in recent years. We provide a comprehensive survey of recent advances in SFDA and organize them into a unified categorization scheme. We compare the results of more than 30 representative SFDA methods on three popular classification benchmarks.
  arXiv  Detail & Related papers  (2023-02-23T06:32:09Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection [79.89082006155135]
  Unsupervised Domain Adaptation (UDA) is an effective approach to tackle the issue of domain shift. UDA methods try to align the source and target representations to improve the generalization on the target domain. The Source-Free Adaptation Domain (SFDA) setting aims to alleviate these concerns by adapting a source-trained model for the target domain without requiring access to the source data.
  arXiv  Detail & Related papers  (2022-03-29T17:50:43Z)
• Style-Guided Domain Adaptation for Face Presentation Attack Detection [21.959450790863432]
  We introduce a novel Style-Guided Domain Adaptation framework for inference-time adaptive PAD. Style-Selective Normalization (SSN) is proposed to explore the domain-specific style information within the high-order feature statistics. The proposed SSN enables the adaptation of the model to the target domain by reducing the style difference between the target and the source domains.
  arXiv  Detail & Related papers  (2022-03-28T08:14:19Z)
• Revisiting Deep Subspace Alignment for Unsupervised Domain Adaptation [42.16718847243166]
  Unsupervised domain adaptation (UDA) aims to transfer and adapt knowledge from a labeled source domain to an unlabeled target domain. Traditionally, subspace-based methods form an important class of solutions to this problem. This paper revisits the use of subspace alignment for UDA and proposes a novel adaptation algorithm that consistently leads to improved generalization.
  arXiv  Detail & Related papers  (2022-01-05T20:16:38Z)
• UMAD: Universal Model Adaptation under Domain and Category Shift [138.12678159620248]
  Universal Model ADaptation (UMAD) framework handles both UDA scenarios without access to source data. We develop an informative consistency score to help distinguish unknown samples from known samples. Experiments on open-set and open-partial-set UDA scenarios demonstrate that UMAD exhibits comparable, if not superior, performance to state-of-the-art data-dependent methods.
  arXiv  Detail & Related papers  (2021-12-16T01:22:59Z)
• Source-Free Open Compound Domain Adaptation in Semantic Segmentation [99.82890571842603]
  In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model. We propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level. Our method produces state-of-the-art results on the C-Driving dataset.
  arXiv  Detail & Related papers  (2021-06-07T08:38:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.