PG-SAM: Prior-Guided SAM with Medical for Multi-organ Segmentation

• URL: http://arxiv.org/abs/2503.18227v3
• Date: Wed, 26 Mar 2025 13:38:40 GMT
• Title: PG-SAM: Prior-Guided SAM with Medical for Multi-organ Segmentation
• Authors: Yiheng Zhong, Zihong Luo, Chengzhi Liu, Feilong Tang, Zelin Peng, Ming Hu, Yingzhen Hu, Jionglong Su, Zongyuan Ge, Imran Razzak,
• Abstract summary: Segment Anything Model (SAM) demonstrates powerful zero-shot capabilities.<n>SAM's accuracy and robustness significantly decrease when applied to medical image segmentation.<n>We propose Prior-Guided SAM (PG-SAM), which employs a fine-grained modality prior aligner.
• Score: 23.98353484144839
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Segment Anything Model (SAM) demonstrates powerful zero-shot capabilities; however, its accuracy and robustness significantly decrease when applied to medical image segmentation. Existing methods address this issue through modality fusion, integrating textual and image information to provide more detailed priors. In this study, we argue that the granularity of text and the domain gap affect the accuracy of the priors. Furthermore, the discrepancy between high-level abstract semantics and pixel-level boundary details in images can introduce noise into the fusion process. To address this, we propose Prior-Guided SAM (PG-SAM), which employs a fine-grained modality prior aligner to leverage specialized medical knowledge for better modality alignment. The core of our method lies in efficiently addressing the domain gap with fine-grained text from a medical LLM. Meanwhile, it also enhances the priors' quality after modality alignment, ensuring more accurate segmentation. In addition, our decoder enhances the model's expressive capabilities through multi-level feature fusion and iterative mask optimizer operations, supporting unprompted learning. We also propose a unified pipeline that effectively supplies high-quality semantic information to SAM. Extensive experiments on the Synapse dataset demonstrate that the proposed PG-SAM achieves state-of-the-art performance. Our code is released at https://github.com/logan-0623/PG-SAM.

Related papers

• Organ-aware Multi-scale Medical Image Segmentation Using Text Prompt Engineering [17.273290949721975]
  Existing medical image segmentation methods rely on uni-modal visual inputs, such as images or videos, requiring labor-intensive manual annotations.<n>Medical imaging techniques capture multiple intertwined organs within a single scan, further complicating segmentation accuracy.<n>To address these challenges, MedSAM was developed to enhance segmentation accuracy by integrating image features with user-provided prompts.
  arXiv  Detail & Related papers  (2025-03-18T01:35:34Z)
• SparseMamba-PCL: Scribble-Supervised Medical Image Segmentation via SAM-Guided Progressive Collaborative Learning [9.228586820098723]
  We propose a Progressive Collaborative Learning framework to enhance information quality during training.<n>We enrich ground truth scribble segmentation labels through a new algorithm, propagating scribbles to estimate object boundaries.<n>We enhance feature representation by optimizing Med-SAM-guided training through the fusion of feature embeddings from Med-SAM and our proposed Sparse Mamba network.
  arXiv  Detail & Related papers  (2025-03-03T15:09:04Z)
• Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond [52.486290612938895]
  Multi-modality image fusion plays a crucial role in integrating diverse modalities to enhance scene understanding.<n>Recent approaches have shifted towards task-specific design, but struggle to achieve the The Best of Both Worlds'' due to inconsistent optimization goals.<n>We propose a novel method that leverages the semantic knowledge from the Segment Anything Model (SAM) to Grow the quality of fusion results and Establish downstream task adaptability, namely SAGE.
  arXiv  Detail & Related papers  (2025-03-03T06:16:31Z)
• DB-SAM: Delving into High Quality Universal Medical Image Segmentation [100.63434169944853]
  We propose a dual-branch adapted SAM framework, named DB-SAM, to bridge the gap between natural and 2D/3D medical data. Our proposed DB-SAM achieves an absolute gain of 8.8%, compared to a recent medical SAM adapter in the literature.
  arXiv  Detail & Related papers  (2024-10-05T14:36:43Z)
• CC-SAM: SAM with Cross-feature Attention and Context for Ultrasound Image Segmentation [20.448864959103858]
  The Segment Anything Model (SAM) has achieved remarkable successes in the realm of natural image segmentation. SAM struggles with medical images that feature low contrast, faint boundaries, intricate morphologies, and small-sized objects. We introduce a comprehensive modification to enhance SAM's performance in the medical domain.
  arXiv  Detail & Related papers  (2024-07-31T22:24:05Z)
• ASPS: Augmented Segment Anything Model for Polyp Segmentation [77.25557224490075]
  The Segment Anything Model (SAM) has introduced unprecedented potential for polyp segmentation. SAM's Transformer-based structure prioritizes global and low-frequency information. CFA integrates a trainable CNN encoder branch with a frozen ViT encoder, enabling the integration of domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-06-30T14:55:32Z)
• MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
  We propose a novel feature learning framework named MAS-SAM for marine animal segmentation. Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
  arXiv  Detail & Related papers  (2024-04-24T07:38:14Z)
• Deep Instruction Tuning for Segment Anything Model [68.7934961590075]
  Segment Anything Model (SAM) has become a research hotspot in the fields of multimedia and computer vision. SAM can support different types of segmentation prompts, but it performs much worse on text-instructed tasks. We propose two simple yet effective deep instruction tuning (DIT) methods for SAM, one is end-to-end and the other is layer-wise.
  arXiv  Detail & Related papers  (2024-03-31T11:37:43Z)
• Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding [15.401507589312702]
  This paper introduces H-SAM, a prompt-free adaptation of the Segment Anything Model (SAM) for efficient fine-tuning of medical images. In the initial stage, H-SAM employs SAM's original decoder to generate a prior probabilistic mask, guiding a more intricate decoding process. Our H-SAM demonstrates a 4.78% improvement in average Dice compared to existing prompt-free SAM variants.
  arXiv  Detail & Related papers  (2024-03-27T05:55:16Z)
• MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation [58.53672866662472]
  We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
  arXiv  Detail & Related papers  (2023-09-16T02:41:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.