Leveraging Segment Anything Model for Source-Free Domain Adaptation via Dual Feature Guided Auto-Prompting

• URL: http://arxiv.org/abs/2505.08527v3
• Date: Sat, 12 Jul 2025 11:00:59 GMT
• Title: Leveraging Segment Anything Model for Source-Free Domain Adaptation via Dual Feature Guided Auto-Prompting
• Authors: Zheang Huai, Hui Tang, Yi Li, Zhuangzhuang Chen, Xiaomeng Li,
• Abstract summary: Source-free domain adaptation (SFDA) for segmentation aims at adapting a model trained in the source domain to perform well in the target domain.<n>We for the first time explore the potentials of Segment Anything Model for SFDA via automatedly finding an accurate bounding box prompt.<n>We propose a novel Dual Feature Guided (DFG) auto-prompting approach to search for the box prompt.
• Score: 14.640695608089796
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Source-free domain adaptation (SFDA) for segmentation aims at adapting a model trained in the source domain to perform well in the target domain with only the source model and unlabeled target data. Inspired by the recent success of Segment Anything Model (SAM) which exhibits the generality of segmenting images of various modalities and in different domains given human-annotated prompts like bounding boxes or points, we for the first time explore the potentials of Segment Anything Model for SFDA via automatedly finding an accurate bounding box prompt. We find that the bounding boxes directly generated with existing SFDA approaches are defective due to the domain gap. To tackle this issue, we propose a novel Dual Feature Guided (DFG) auto-prompting approach to search for the box prompt. Specifically, the source model is first trained in a feature aggregation phase, which not only preliminarily adapts the source model to the target domain but also builds a feature distribution well-prepared for box prompt search. In the second phase, based on two feature distribution observations, we gradually expand the box prompt with the guidance of the target model feature and the SAM feature to handle the class-wise clustered target features and the class-wise dispersed target features, respectively. To remove the potentially enlarged false positive regions caused by the over-confident prediction of the target model, the refined pseudo-labels produced by SAM are further postprocessed based on connectivity analysis. Experiments on 3D and 2D datasets indicate that our approach yields superior performance compared to conventional methods. Code is available at https://github.com/xmed-lab/DFG.

Related papers

• Adapter Naturally Serves as Decoupler for Cross-Domain Few-Shot Semantic Segmentation [14.660710170156202]
  Cross-domain few-shot segmentation (CD-FSS) is proposed to pre-train the model on a source-domain dataset with sufficient samples.<n>On target domains, we freeze the model and fine-tune the DFN to learn target-specific knowledge specific.<n>Our method surpasses the state-of-the-art method in CD-FSS significantly by 2.69% and 4.68% MIoU in 1-shot and 5-shot scenarios.
  arXiv  Detail & Related papers  (2025-06-09T02:51:06Z)
• High-order Neighborhoods Know More: HyperGraph Learning Meets Source-free Unsupervised Domain Adaptation [34.08681468394247]
  Source-free Unsupervised Domain Adaptation aims to classify target samples by only accessing a pre-trained source model and unlabelled target samples. Existing methods normally exploit the pair-wise relation among target samples and attempt to discover their correlations by clustering these samples based on semantic features. We propose a new SFDA method that exploits the high-order neighborhood relation and explicitly takes the domain shift effect into account.
  arXiv  Detail & Related papers  (2024-05-11T05:07:43Z)
• CNG-SFDA:Clean-and-Noisy Region Guided Online-Offline Source-Free Domain Adaptation [6.222371087167951]
  Source-Free Domain Adaptation (SFDA) aims to adopt a trained model on the source domain to the target domain. handling false labels in the target domain is crucial because they negatively impact the model performance. We conduct extensive experiments on multiple datasets in online/offline SFDA settings, whose results demonstrate that our method, CNG-SFDA, achieves state-of-the-art for most cases.
  arXiv  Detail & Related papers  (2024-01-26T01:29:37Z)
• Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation [80.1412989006262]
  Domain adaptive semantic segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain. We propose T2S-DA, which we interpret as a form of pulling Target to Source for Domain Adaptation.
  arXiv  Detail & Related papers  (2023-05-23T07:09:09Z)
• Semi-supervised Domain Adaptation via Prototype-based Multi-level Learning [4.232614032390374]
  In semi-supervised domain adaptation (SSDA), a few labeled target samples of each class help the model to transfer knowledge representation from the fully labeled source domain to the target domain. We propose a Prototype-based Multi-level Learning (ProML) framework to better tap the potential of labeled target samples.
  arXiv  Detail & Related papers  (2023-05-04T10:09:30Z)
• Robust Target Training for Multi-Source Domain Adaptation [110.77704026569499]
  We propose a novel Bi-level Optimization based Robust Target Training (BORT$2$) method for MSDA. Our proposed method achieves the state of the art performance on three MSDA benchmarks, including the large-scale DomainNet dataset.
  arXiv  Detail & Related papers  (2022-10-04T15:20:01Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Source-Free Open Compound Domain Adaptation in Semantic Segmentation [99.82890571842603]
  In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model. We propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level. Our method produces state-of-the-art results on the C-Driving dataset.
  arXiv  Detail & Related papers  (2021-06-07T08:38:41Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)
• Casting a BAIT for Offline and Online Source-free Domain Adaptation [51.161476418834766]
  We address the source-free domain adaptation (SFDA) problem, where only the source model is available during adaptation to the target domain. Inspired by diverse classifier based domain adaptation methods, in this paper we introduce a second classifier. When adapting to the target domain, the additional classifier from source is expected to find misclassified features. Our method surpasses by a large margin other SFDA methods under online source-free domain adaptation setting.
  arXiv  Detail & Related papers  (2020-10-23T14:18:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.