When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts

• URL: http://arxiv.org/abs/2507.14661v1
• Date: Sat, 19 Jul 2025 15:18:28 GMT
• Title: When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts
• Authors: Wooseok Ha, Yuansi Chen,
• Abstract summary: Semi-supervised domain adaptation (SSDA) aims to achieve high predictive performance in the target domain with limited labeled target data.<n>We develop a theoretical framework based on structural causal models (SCMs) which allows us to analyze and quantify the performance of SSDA methods when labeled target data is limited.<n>We propose the Multi Adaptive-Start Fine-Tuning (MASFT) algorithm, which fine-tunes UDA models from multiple starting points and selects the best-performing one.
• Score: 5.839411310096219
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semi-supervised domain adaptation (SSDA) aims to achieve high predictive performance in the target domain with limited labeled target data by exploiting abundant source and unlabeled target data. Despite its significance in numerous applications, theory on the effectiveness of SSDA remains largely unexplored, particularly in scenarios involving various types of source-target distributional shifts. In this work, we develop a theoretical framework based on structural causal models (SCMs) which allows us to analyze and quantify the performance of SSDA methods when labeled target data is limited. Within this framework, we introduce three SSDA methods, each having a fine-tuning strategy tailored to a distinct assumption about the source and target relationship. Under each assumption, we demonstrate how extending an unsupervised domain adaptation (UDA) method to SSDA can achieve minimax-optimal target performance with limited target labels. When the relationship between source and target data is only vaguely known -- a common practical concern -- we propose the Multi Adaptive-Start Fine-Tuning (MASFT) algorithm, which fine-tunes UDA models from multiple starting points and selects the best-performing one based on a small hold-out target validation dataset. Combined with model selection guarantees, MASFT achieves near-optimal target predictive performance across a broad range of types of distributional shifts while significantly reducing the need for labeled target data. We empirically validate the effectiveness of our proposed methods through simulations.

Related papers

• Progressive Multi-Source Domain Adaptation for Personalized Facial Expression Recognition [51.61979855488214]
  Personalized facial expression recognition (FER) involves adapting a machine learning model using samples from labeled sources and unlabeled target domains.<n>We propose a progressive MSDA approach that gradually introduces information from subjects based on their similarity to the target subject.<n>Our experiments show the effectiveness of our proposed method on pain datasets: Biovid and UNBC-McMaster.
  arXiv  Detail & Related papers  (2025-04-05T19:14:51Z)
• Asymmetric Co-Training for Source-Free Few-Shot Domain Adaptation [5.611768906855499]
  We propose an asymmetric co-training (ACT) method specifically designed for the SFFSDA scenario.<n>We use a two-step optimization process to train the target model.<n>Our findings suggest that adapting a source pre-trained model using only a small amount of labeled target data offers a practical and dependable solution.
  arXiv  Detail & Related papers  (2025-02-20T02:58:45Z)
• CAusal and collaborative proxy-tasKs lEarning for Semi-Supervised Domain Adaptation [20.589323508870592]
  Semi-supervised domain adaptation (SSDA) adapts a learner to a new domain by effectively utilizing source domain data and a few labeled target samples. We show that the proposed model significantly outperforms SOTA methods in terms of effectiveness and generalisability on SSDA datasets.
  arXiv  Detail & Related papers  (2023-03-30T16:48:28Z)
• MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation [98.09845149258972]
  We introduce active sample selection to assist domain adaptation regarding the semantic segmentation task. With only a little workload to manually annotate these samples, the distortion of the target-domain distribution can be effectively alleviated. A powerful semi-supervised domain adaptation strategy is proposed to alleviate the long-tail distribution problem.
  arXiv  Detail & Related papers  (2023-01-18T07:55:22Z)
• Domain-Specific Risk Minimization for Out-of-Distribution Generalization [104.17683265084757]
  We first establish a generalization bound that explicitly considers the adaptivity gap. We propose effective gap estimation methods for guiding the selection of a better hypothesis for the target. The other method is minimizing the gap directly by adapting model parameters using online target samples.
  arXiv  Detail & Related papers  (2022-08-18T06:42:49Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Learning Invariant Representation with Consistency and Diversity for Semi-supervised Source Hypothesis Transfer [46.68586555288172]
  We propose a novel task named Semi-supervised Source Hypothesis Transfer (SSHT), which performs domain adaptation based on source trained model, to generalize well in target domain with a few supervisions. We propose Consistency and Diversity Learning (CDL), a simple but effective framework for SSHT by facilitating prediction consistency between two randomly augmented unlabeled data. Experimental results show that our method outperforms existing SSDA methods and unsupervised model adaptation methods on DomainNet, Office-Home and Office-31 datasets.
  arXiv  Detail & Related papers  (2021-07-07T04:14:24Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Dynamic Domain Adaptation for Efficient Inference [12.713628738434881]
  Domain adaptation (DA) enables knowledge transfer from a labeled source domain to an unlabeled target domain. Most prior DA approaches leverage complicated and powerful deep neural networks to improve the adaptation capacity. We propose a dynamic domain adaptation (DDA) framework, which can simultaneously achieve efficient target inference in low-resource scenarios.
  arXiv  Detail & Related papers  (2021-03-26T08:53:16Z)
• Towards Inheritable Models for Open-Set Domain Adaptation [56.930641754944915]
  We introduce a practical Domain Adaptation paradigm where a source-trained model is used to facilitate adaptation in the absence of the source dataset in future. We present an objective way to quantify inheritability to enable the selection of the most suitable source model for a given target domain, even in the absence of the source data.
  arXiv  Detail & Related papers  (2020-04-09T07:16:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.