Fully Automated SAM for Single-source Domain Generalization in Medical Image Segmentation

• URL: http://arxiv.org/abs/2507.17281v1
• Date: Wed, 23 Jul 2025 07:37:39 GMT
• Title: Fully Automated SAM for Single-source Domain Generalization in Medical Image Segmentation
• Authors: Huanli Zhuo, Leilei Ma, Haifeng Zhao, Shiwei Zhou, Dengdi Sun, Yanping Fu,
• Abstract summary: FA-SAM is a single-source domain generalization framework for medical image segmentation that achieves fully automated SAM.<n>FA-SAM introduces two key innovations: an Auto-prompted Generation Model (AGM) branch equipped with a Shallow Feature Uncertainty Modeling (SUFM) module, and an Image-Prompt Embedding Fusion (IPEF) module integrated into the SAM mask decoder.
• Score: 3.7839630649682054
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Although SAM-based single-source domain generalization models for medical image segmentation can mitigate the impact of domain shift on the model in cross-domain scenarios, these models still face two major challenges. First, the segmentation of SAM is highly dependent on domain-specific expert-annotated prompts, which prevents SAM from achieving fully automated medical image segmentation and therefore limits its application in clinical settings. Second, providing poor prompts (such as bounding boxes that are too small or too large) to the SAM prompt encoder can mislead SAM into generating incorrect mask results. Therefore, we propose the FA-SAM, a single-source domain generalization framework for medical image segmentation that achieves fully automated SAM. FA-SAM introduces two key innovations: an Auto-prompted Generation Model (AGM) branch equipped with a Shallow Feature Uncertainty Modeling (SUFM) module, and an Image-Prompt Embedding Fusion (IPEF) module integrated into the SAM mask decoder. Specifically, AGM models the uncertainty distribution of shallow features through the SUFM module to generate bounding box prompts for the target domain, enabling fully automated segmentation with SAM. The IPEF module integrates multiscale information from SAM image embeddings and prompt embeddings to capture global and local details of the target object, enabling SAM to mitigate the impact of poor prompts. Extensive experiments on publicly available prostate and fundus vessel datasets validate the effectiveness of FA-SAM and highlight its potential to address the above challenges.

Related papers

• SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation [88.80792308991867]
  Segment Anything model (SAM) has shown ability to group image pixels into patches, but applying it to semantic-aware segmentation still faces major challenges.<n>This paper presents SAM-CP, a simple approach that establishes two types of composable prompts beyond SAM and composes them for versatile segmentation.<n> Experiments show that SAM-CP achieves semantic, instance, and panoptic segmentation in both open and closed domains.
  arXiv  Detail & Related papers  (2024-07-23T17:47:25Z)
• AlignSAM: Aligning Segment Anything Model to Open Context via Reinforcement Learning [61.666973416903005]
  Segment Anything Model (SAM) has demonstrated its impressive generalization capabilities in open-world scenarios with the guidance of prompts. We propose a novel framework, termed AlignSAM, designed for automatic prompting for aligning SAM to an open context.
  arXiv  Detail & Related papers  (2024-06-01T16:21:39Z)
• MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
  We propose a novel feature learning framework named MAS-SAM for marine animal segmentation. Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
  arXiv  Detail & Related papers  (2024-04-24T07:38:14Z)
• Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding [15.401507589312702]
  This paper introduces H-SAM, a prompt-free adaptation of the Segment Anything Model (SAM) for efficient fine-tuning of medical images. In the initial stage, H-SAM employs SAM's original decoder to generate a prior probabilistic mask, guiding a more intricate decoding process. Our H-SAM demonstrates a 4.78% improvement in average Dice compared to existing prompt-free SAM variants.
  arXiv  Detail & Related papers  (2024-03-27T05:55:16Z)
• WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images [8.179859593451285]
  We present WSI-SAM, enhancing Segment Anything Model (SAM) with precise object segmentation capabilities for histopathology images. To fully exploit pretrained knowledge while minimizing training overhead, we keep SAM frozen, introducing only minimal extra parameters. Our model outperforms SAM by 4.1 and 2.5 percent points on a ductal carcinoma in situ (DCIS) segmentation tasks and breast cancer metastasis segmentation task.
  arXiv  Detail & Related papers  (2024-03-14T10:30:43Z)
• UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images [47.59627416801523]
  In digital pathology, precise nuclei segmentation is pivotal yet challenged by the diversity of tissue types, staining protocols, and imaging conditions. We propose the Universal prompt-free SAM framework for Nuclei segmentation (UN-SAM) UN-SAM with exceptional performance surpasses state-of-the-arts in nuclei instance and semantic segmentation, especially the generalization capability in zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-02-26T15:35:18Z)
• Promoting Segment Anything Model towards Highly Accurate Dichotomous Image Segmentation [11.133337712285424]
  We propose DIS-SAM, which advances the Segment Anything Model (SAM) towards highly accurate object segmentation.<n> DIS-SAM employs a two-stage approach, integrating SAM with a modified advanced network that was previously designed to handle the prompt-free DIS task.<n>Despite its simplicity, DIS-SAM significantly advances the SAM, HQ-SAM, and Pi-SAM by 8.5%, 6.9%, and 3.7% maximum F-measure.
  arXiv  Detail & Related papers  (2023-12-30T14:24:33Z)
• Boosting Segment Anything Model Towards Open-Vocabulary Learning [69.24734826209367]
  Segment Anything Model (SAM) has emerged as a new paradigmatic vision foundation model.<n>Despite SAM finding applications and adaptations in various domains, its primary limitation lies in the inability to grasp object semantics.<n>We present Sambor to seamlessly integrate SAM with the open-vocabulary object detector in an end-to-end framework.
  arXiv  Detail & Related papers  (2023-12-06T17:19:00Z)
• Stable Segment Anything Model [79.9005670886038]
  The Segment Anything Model (SAM) achieves remarkable promptable segmentation given high-quality prompts. This paper presents the first comprehensive analysis on SAM's segmentation stability across a diverse spectrum of prompt qualities. Our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality.
  arXiv  Detail & Related papers  (2023-11-27T12:51:42Z)
• Beyond Adapting SAM: Towards End-to-End Ultrasound Image Segmentation via Auto Prompting [10.308637269138146]
  We propose SAMUS as a universal model tailored for ultrasound image segmentation. We further enable it to work in an end-to-end manner denoted as AutoSAMUS. AutoSAMUS is realized by introducing an auto prompt generator (APG) to replace the manual prompt encoder of SAMUS.
  arXiv  Detail & Related papers  (2023-09-13T09:15:20Z)
• SAM Meets Robotic Surgery: An Empirical Study on Generalization, Robustness and Adaptation [15.995869434429274]
  The Segment Anything Model (SAM) serves as a fundamental model for semantic segmentation. We examine SAM's robustness and zero-shot generalizability in the field of robotic surgery.
  arXiv  Detail & Related papers  (2023-08-14T14:09:41Z)
• DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation [22.974876391669685]
  Segment Anything Model (SAM) shows potential for improving the cross-domain robustness of medical image segmentation. SAM performs significantly worse in automatic segmentation scenarios than when manually prompted. Decoupled SAM modifies SAM's mask decoder by introducing two new modules.
  arXiv  Detail & Related papers  (2023-06-01T09:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.