ProMISe: Promptable Medical Image Segmentation using SAM

• URL: http://arxiv.org/abs/2403.04164v3
• Date: Sat, 28 Sep 2024 12:59:54 GMT
• Title: ProMISe: Promptable Medical Image Segmentation using SAM
• Authors: Jinfeng Wang, Sifan Song, Xinkun Wang, Yiyi Wang, Yiyi Miao, Jionglong Su, S. Kevin Zhou,
• Abstract summary: We propose an Auto-Prompting Module (APM) which provides SAM-based foundation model with Euclidean adaptive prompts in the target domain. We also propose a novel non-invasive method called Incremental Pattern Shifting (IPS) to adapt SAM to specific medical domains. By coupling these two methods, we propose ProMISe, an end-to-end non-fine-tuned framework for Promptable Medical Image.
• Score: 11.710367186709432
• License:
• Abstract: With the proposal of the Segment Anything Model (SAM), fine-tuning SAM for medical image segmentation (MIS) has become popular. However, due to the large size of the SAM model and the significant domain gap between natural and medical images, fine-tuning-based strategies are costly with potential risk of instability, feature damage and catastrophic forgetting. Furthermore, some methods of transferring SAM to a domain-specific MIS through fine-tuning strategies disable the model's prompting capability, severely limiting its utilization scenarios. In this paper, we propose an Auto-Prompting Module (APM), which provides SAM-based foundation model with Euclidean adaptive prompts in the target domain. Our experiments demonstrate that such adaptive prompts significantly improve SAM's non-fine-tuned performance in MIS. In addition, we propose a novel non-invasive method called Incremental Pattern Shifting (IPS) to adapt SAM to specific medical domains. Experimental results show that the IPS enables SAM to achieve state-of-the-art or competitive performance in MIS without the need for fine-tuning. By coupling these two methods, we propose ProMISe, an end-to-end non-fine-tuned framework for Promptable Medical Image Segmentation. Our experiments demonstrate that both using our methods individually or in combination achieves satisfactory performance in low-cost pattern shifting, with all of SAM's parameters frozen.

Related papers

• Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation [52.172885882728174]
  In medical imaging contexts, it is not uncommon for human experts to rectify segmentations of specific test samples after SAM generates its segmentation predictions. We introduce a novel approach that leverages the advantages of online machine learning to enhance Segment Anything (SA) during test time. We employ rectified annotations to perform online learning, with the aim of improving the segmentation quality of SA on medical images.
  arXiv  Detail & Related papers  (2024-06-03T03:16:25Z)
• Segment Any Medical Model Extended [39.80956010574076]
  We introduce SAMM Extended (SAMME), a platform that integrates new SAM variant models, adopts faster communication protocols, accommodates new interactive modes, and allows for fine-tuning of subcomponents of the models. These features can expand the potential of foundation models like SAM, and the results can be translated to applications such as image-guided therapy, mixed reality interaction, robotic navigation, and data augmentation.
  arXiv  Detail & Related papers  (2024-03-26T21:37:25Z)
• Uncertainty-Aware Adapter: Adapting Segment Anything Model (SAM) for Ambiguous Medical Image Segmentation [20.557472889654758]
  The Segment Anything Model (SAM) gained significant success in natural image segmentation. Unlike natural images, many tissues and lesions in medical images have blurry boundaries and may be ambiguous. We propose a novel module called the Uncertainty-aware Adapter, which efficiently fine-tune SAM for uncertainty-aware medical image segmentation.
  arXiv  Detail & Related papers  (2024-03-16T14:11:54Z)
• Stable Segment Anything Model [79.9005670886038]
  The Segment Anything Model (SAM) achieves remarkable promptable segmentation given high-quality prompts. This paper presents the first comprehensive analysis on SAM's segmentation stability across a diverse spectrum of prompt qualities. Our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality.
  arXiv  Detail & Related papers  (2023-11-27T12:51:42Z)
• SAMIHS: Adaptation of Segment Anything Model for Intracranial Hemorrhage Segmentation [18.867207134086193]
  Intracranial hemorrhage segmentation is a crucial and challenging step in stroke diagnosis and surgical planning. We propose a SAM-based parameter-efficient fine-tuning method, called SAMIHS, for intracranial hemorrhage segmentation. Our experimental results on two public datasets demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-11-14T14:23:09Z)
• SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbation [61.732503554088524]
  We investigate whether it is possible to attack Segment Anything Model (SAM) with image-aversagnostic Universal Adrial Perturbation (UAP) We propose a novel perturbation-centric framework that results in a UAP generation method based on self-supervised contrastive learning (CL) The effectiveness of our proposed CL-based UAP generation method is validated by both quantitative and qualitative results.
  arXiv  Detail & Related papers  (2023-10-19T02:49:24Z)
• Cheap Lunch for Medical Image Segmentation by Fine-tuning SAM on Few Exemplars [19.725817146049707]
  The Segment Anything Model (SAM) has demonstrated remarkable capabilities of scaled-up segmentation models. However, the adoption of foundational models in the medical domain presents a challenge due to the difficulty and expense of labeling sufficient data. This paper introduces an efficient and practical approach for fine-tuning SAM using a limited number of exemplars.
  arXiv  Detail & Related papers  (2023-08-27T15:21:25Z)
• SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation [65.52097667738884]
  We introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to integrate surgical-specific information with SAM's pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning.
  arXiv  Detail & Related papers  (2023-08-17T02:51:01Z)
• AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation [49.59991322513561]
  We propose an adaptive modification of Segment-Anything (SAM) that can adjust to new datasets quickly and efficiently. AdaptiveSAM uses free-form text as prompt and can segment the object of interest with just the label name as prompt. Our experiments show that AdaptiveSAM outperforms current state-of-the-art methods on various medical imaging datasets.
  arXiv  Detail & Related papers  (2023-08-07T17:12:54Z)
• Temporally-Extended Prompts Optimization for SAM in Interactive Medical Image Segmentation [22.242652304004068]
  The Anything Model (SAM) has emerged as a foundation model for addressing image segmentation. This paper focuses on assessing the potential of SAM's zero-shot capabilities within the interactive medical image segmentation (IMIS) paradigm. We develop a framework that adaptively offers suitable prompt forms for human experts.
  arXiv  Detail & Related papers  (2023-06-15T08:51:24Z)
• Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation [51.770805270588625]
  The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation. Recent studies and individual experiments have shown that SAM underperforms in medical image segmentation. We propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model.
  arXiv  Detail & Related papers  (2023-04-25T07:34:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.