Few-shot Fine-tuning is All You Need for Source-free Domain Adaptation
- URL: http://arxiv.org/abs/2304.00792v2
- Date: Mon, 24 Apr 2023 13:23:53 GMT
- Title: Few-shot Fine-tuning is All You Need for Source-free Domain Adaptation
- Authors: Suho Lee, Seungwon Seo, Jihyo Kim, Yejin Lee, Sangheum Hwang
- Abstract summary: We investigate the practicality of source-free unsupervised domain adaptation (SFUDA) over unsupervised domain adaptation (UDA)
We show that SFUDA relies on unlabeled target data, which limits its practicality in real-world applications.
We show that fine-tuning a source pretrained model with a few labeled data is a practical and reliable solution to circumvent the limitations of SFUDA.
- Score: 2.837894907597713
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, source-free unsupervised domain adaptation (SFUDA) has emerged as a
more practical and feasible approach compared to unsupervised domain adaptation
(UDA) which assumes that labeled source data are always accessible. However,
significant limitations associated with SFUDA approaches are often overlooked,
which limits their practicality in real-world applications. These limitations
include a lack of principled ways to determine optimal hyperparameters and
performance degradation when the unlabeled target data fail to meet certain
requirements such as a closed-set and identical label distribution to the
source data. All these limitations stem from the fact that SFUDA entirely
relies on unlabeled target data. We empirically demonstrate the limitations of
existing SFUDA methods in real-world scenarios including out-of-distribution
and label distribution shifts in target data, and verify that none of these
methods can be safely applied to real-world settings. Based on our experimental
results, we claim that fine-tuning a source pretrained model with a few labeled
data (e.g., 1- or 3-shot) is a practical and reliable solution to circumvent
the limitations of SFUDA. Contrary to common belief, we find that carefully
fine-tuned models do not suffer from overfitting even when trained with only a
few labeled data, and also show little change in performance due to sampling
bias. Our experimental results on various domain adaptation benchmarks
demonstrate that the few-shot fine-tuning approach performs comparatively under
the standard SFUDA settings, and outperforms comparison methods under realistic
scenarios. Our code is available at https://github.com/daintlab/fewshot-SFDA .
Related papers
- Asymmetric Co-Training for Source-Free Few-Shot Domain Adaptation [5.611768906855499]
We propose an asymmetric co-training (ACT) method specifically designed for the SFFSDA scenario.
We use a two-step optimization process to train the target model.
Our findings suggest that adapting a source pre-trained model using only a small amount of labeled target data offers a practical and dependable solution.
arXiv Detail & Related papers (2025-02-20T02:58:45Z) - Better Practices for Domain Adaptation [62.70267990659201]
Domain adaptation (DA) aims to provide frameworks for adapting models to deployment data without using labels.
Unclear validation protocol for DA has led to bad practices in the literature.
We show challenges across all three branches of domain adaptation methodology.
arXiv Detail & Related papers (2023-09-07T17:44:18Z) - SF-FSDA: Source-Free Few-Shot Domain Adaptive Object Detection with
Efficient Labeled Data Factory [94.11898696478683]
Domain adaptive object detection aims to leverage the knowledge learned from a labeled source domain to improve the performance on an unlabeled target domain.
We propose and investigate a more practical and challenging domain adaptive object detection problem under both source-free and few-shot conditions, named as SF-FSDA.
arXiv Detail & Related papers (2023-06-07T12:34:55Z) - Divide and Contrast: Source-free Domain Adaptation via Adaptive
Contrastive Learning [122.62311703151215]
Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations.
DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals.
We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
arXiv Detail & Related papers (2022-11-12T09:21:49Z) - Source-Free Unsupervised Domain Adaptation with Norm and Shape
Constraints for Medical Image Segmentation [0.12183405753834559]
We propose a source-free unsupervised domain adaptation (SFUDA) method for medical image segmentation.
In addition to the entropy minimization method, we introduce a loss function for avoiding feature norms in the target domain small.
Our method outperforms the state-of-the-art in all datasets.
arXiv Detail & Related papers (2022-09-03T00:16:39Z) - Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions.
In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data.
We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
arXiv Detail & Related papers (2022-01-11T23:01:12Z) - Open-Set Hypothesis Transfer with Semantic Consistency [99.83813484934177]
We introduce a method that focuses on the semantic consistency under transformation of target data.
Our model first discovers confident predictions and performs classification with pseudo-labels.
As a result, unlabeled data can be classified into discriminative classes coincided with either source classes or unknown classes.
arXiv Detail & Related papers (2020-10-01T10:44:31Z) - Do We Really Need to Access the Source Data? Source Hypothesis Transfer
for Unsupervised Domain Adaptation [102.67010690592011]
Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain.
Prior UDA methods typically require to access the source data when learning to adapt the model.
This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
arXiv Detail & Related papers (2020-02-20T03:13:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.