Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning

• URL: http://arxiv.org/abs/2605.11659v1
• Date: Tue, 12 May 2026 07:20:23 GMT
• Title: Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning
• Abstract summary: Cross-Domain Few-Shot Learning aims to adapt large-scale pretrained models to specialized target domains with limited samples.<n>We find adapter-based methods consistently outperform prompt-based ones, contrary to in-domain scenarios.<n>We propose Semantic Probe, a plug-and-play attention rectification framework for both adapter- and prompt-based methods.
• Score: 19.4465469479799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-Domain Few-Shot Learning (CDFSL) aims to adapt large-scale pretrained models to specialized target domains with limited samples, yet the few-shot fine-tuning of vision-language models like CLIP remains underexplored. By establishing multiple fine-tuning baselines of CLIP for CDFSL, we find adapter-based methods (e.g., LoRA) consistently outperform prompt-based ones (e.g., MaPLe), contrary to in-domain scenarios. To make those effective in-domain methods competitive again in CDFSL, we analyze this phenomenon and discover LoRA's superiority stems from rectifying the collapsed attention of visual CLS token, enhancing modality alignment and class separation by focusing on text-related visual regions. Further, we find textual EOS token exhibit much better attention to visual samples, and CLIP's standard contrastive loss weakly constrains modality alignment. Based on these insights, we propose Semantic Probe, a plug-and-play attention rectification framework for both adapter- and prompt-based methods. Extensive experiments on four CDFSL benchmarks validate our rationale, achieving state-of-the-art performance and benefiting both fine-tuning paradigms. Codes will be released.

Related papers

• Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning [25.20062959668559]
  This paper focuses on the target-domain few-shot finetuning in the CLIP-based CDFSL task.<n>We find a counterintuitive phenomenon: actively pushing away certain low-similarity image tokens improves target-domain performance.<n>Motivated by this, we propose Adaptive Tail-Head Alignment (ATHA), a novel fine-tuning strategy for CLIP.
  arXiv  Detail & Related papers  (2026-05-28T11:21:44Z)
• Addressing Exacerbated Attention Sink for Source-Free Cross-Domain Few-Shot Learning [25.20062959668559]
  Cross-Domain Few-Shot Learning can transfer source-domain information to target domains with scarce training data.<n>Standard target-domain few-shot fine-tuning exacerbates the attention sink problem, leading to poor discriminability across classes.<n>We propose a novel approach to dynamically re-weight tokens according to their relevance with target-domain classes during the target-domain finetuning.
  arXiv  Detail & Related papers  (2026-05-25T12:49:15Z)
• Interpretable Cross-Domain Few-Shot Learning with Rectified Target-Domain Local Alignment [19.113214017897118]
  Cross-Domain Few-Shot Learning adapts models trained with large-scale general data (source domain) to downstream target domains with only scarce training data.<n>We find that CLIP models can hardly focus on fine-grained visual cues for interpretable recognition.<n>To address this problem, due to the lack of supervision in aligning local visual features and text semantics, we turn to self-supervision information.
  arXiv  Detail & Related papers  (2026-03-18T12:20:21Z)
• Mind the Discriminability Trap in Source-Free Cross-domain Few-shot Learning [30.80780619903459]
  Source-Free Cross-Domain Few-Shot Learning focuses on fine-tuning with limited training data from target domains.<n>We find that improving visual discriminability actually suppresses VLMs' performance.<n>We propose an approach to address this problem: first, we perturb the visual learning to guide the model to focus on the cross-modal alignment.
  arXiv  Detail & Related papers  (2026-03-07T03:59:23Z)
• Multi-Prompt Progressive Alignment for Multi-Source Unsupervised Domain Adaptation [73.40696661117408]
  We propose a progressive alignment strategy for adapting CLIP to unlabeled downstream task.<n>We name our approach MP2A and test it on three popular UDA benchmarks, namely ImageCLEF, Office-Home, and the most challenging DomainNet.<n> Experiments showcase that MP2A achieves state-of-the-art performance when compared with most recent CLIP-based MS-UDA approaches.
  arXiv  Detail & Related papers  (2025-07-31T09:42:42Z)
• Dual Guidance Semi-Supervised Action Detection [71.45023660211145]
  We present a semi-supervised approach for spatial-temporal action localization.<n>We introduce a dual guidance network to select better pseudo-bounding boxes.<n>Our framework achieves superior results compared to extended image-based semi-supervised baselines.
  arXiv  Detail & Related papers  (2025-07-28T18:08:36Z)
• Multiple Stochastic Prompt Tuning for Few-shot Adaptation under Extreme Domain Shift [14.85375816073596]
  We introduce multiple learnable prompts per class to capture diverse modes in visual representations arising from distribution shifts.<n>These prompts are modeled as learnable Gaussian distributions, enabling efficient exploration of the prompt parameter space.<n>Experiments and comparisons with state-of-the-art methods demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2025-06-04T13:18:04Z)
• UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models [75.77651291095565]
  We leverage unlabeled data that naturally spans multiple domains to enhance the transferability of vision-language models. Under this unsupervised multi-domain setting, we have identified inherent model bias within CLIP. We propose Unsupervised Multi-domain Feature (UMFC) to mitigate this model bias.
  arXiv  Detail & Related papers  (2024-11-11T12:25:02Z)
• In the Era of Prompt Learning with Vision-Language Models [1.060608983034705]
  We introduce textscStyLIP, a novel domain-agnostic prompt learning strategy for Domain Generalization (DG) StyLIP disentangles visual style and content in CLIPs vision encoder by using style projectors to learn domain-specific prompt tokens. We also propose AD-CLIP for unsupervised domain adaptation (DA), leveraging CLIPs frozen vision backbone.
  arXiv  Detail & Related papers  (2024-11-07T17:31:21Z)
• Transferrable Contrastive Learning for Visual Domain Adaptation [108.98041306507372]
  Transferrable Contrastive Learning (TCL) is a self-supervised learning paradigm tailored for domain adaptation. TCL penalizes cross-domain intra-class domain discrepancy between source and target through a clean and novel contrastive loss. The free lunch is, thanks to the incorporation of contrastive learning, TCL relies on a moving-averaged key encoder that naturally achieves a temporally ensembled version of pseudo labels for target data.
  arXiv  Detail & Related papers  (2021-12-14T16:23:01Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis [120.75020271706978]
  Few-shot learning uses prior knowledge learned from the seen classes to recognize the unseen classes. This class-difference-caused distribution shift can be considered as a special case of domain shift. We propose a prototypical domain adaptation network with attention (DAPNA) to explicitly tackle such a domain shift problem in a meta-learning framework.
  arXiv  Detail & Related papers  (2020-02-06T01:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.