CoSAM: Self-Correcting SAM for Domain Generalization in 2D Medical Image Segmentation

• URL: http://arxiv.org/abs/2411.10136v1
• Date: Fri, 15 Nov 2024 12:20:52 GMT
• Title: CoSAM: Self-Correcting SAM for Domain Generalization in 2D Medical Image Segmentation
• Authors: Yihang Fu, Ziyang Chen, Yiwen Ye, Xingliang Lei, Zhisong Wang, Yong Xia,
• Abstract summary: We propose a method for 2D medical image segmentation called Self-Correcting SAM (CoSAM) Our approach begins by generating coarse masks using SAM in a prompt-free manner, providing prior prompts for the subsequent stages. We generate diverse prompts as feedback based on the corrected masks, which are used to iteratively refine the predictions.
• Score: 11.567414253208991
• License:
• Abstract: Medical images often exhibit distribution shifts due to variations in imaging protocols and scanners across different medical centers. Domain Generalization (DG) methods aim to train models on source domains that can generalize to unseen target domains. Recently, the segment anything model (SAM) has demonstrated strong generalization capabilities due to its prompt-based design, and has gained significant attention in image segmentation tasks. Existing SAM-based approaches attempt to address the need for manual prompts by introducing prompt generators that automatically generate these prompts. However, we argue that auto-generated prompts may not be sufficiently accurate under distribution shifts, potentially leading to incorrect predictions that still require manual verification and correction by clinicians. To address this challenge, we propose a method for 2D medical image segmentation called Self-Correcting SAM (CoSAM). Our approach begins by generating coarse masks using SAM in a prompt-free manner, providing prior prompts for the subsequent stages, and eliminating the need for prompt generators. To automatically refine these coarse masks, we introduce a generalized error decoder that simulates the correction process typically performed by clinicians. Furthermore, we generate diverse prompts as feedback based on the corrected masks, which are used to iteratively refine the predictions within a self-correcting loop, enhancing the generalization performance of our model. Extensive experiments on two medical image segmentation benchmarks across multiple scenarios demonstrate the superiority of CoSAM over state-of-the-art SAM-based methods.

Related papers

• SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting [6.739803086387235]
  Medical image segmentation often faces the challenge of prohibitively expensive annotation costs. We propose leveraging the Segment Anything Model (SAM), pre-trained on over 1 billion masks. We develop SAM-MPA, an innovative SAM-based framework for few-shot medical image segmentation.
  arXiv  Detail & Related papers  (2024-11-26T12:12:12Z)
• Med-PerSAM: One-Shot Visual Prompt Tuning for Personalized Segment Anything Model in Medical Domain [30.700648813505158]
  Leveraging pre-trained models with tailored prompts for in-context learning has proven highly effective in NLP tasks. We introduce textbfMed-PerSAM, a novel and straightforward one-shot framework designed for the medical domain. Our model outperforms various foundational models and previous SAM-based approaches across diverse 2D medical imaging datasets.
  arXiv  Detail & Related papers  (2024-11-25T06:16:17Z)
• CycleSAM: One-Shot Surgical Scene Segmentation using Cycle-Consistent Feature Matching to Prompt SAM [2.9500242602590565]
  CycleSAM is an approach for one-shot surgical scene segmentation using the training image-mask pair at test-time. We employ a ResNet50 encoder pretrained on surgical images in a self-supervised fashion, thereby maintaining high label-efficiency.
  arXiv  Detail & Related papers  (2024-07-09T12:08:07Z)
• Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation [52.172885882728174]
  In medical imaging contexts, it is not uncommon for human experts to rectify segmentations of specific test samples after SAM generates its segmentation predictions. We introduce a novel approach that leverages the advantages of online machine learning to enhance Segment Anything (SA) during test time. We employ rectified annotations to perform online learning, with the aim of improving the segmentation quality of SA on medical images.
  arXiv  Detail & Related papers  (2024-06-03T03:16:25Z)
• Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding [15.401507589312702]
  This paper introduces H-SAM, a prompt-free adaptation of the Segment Anything Model (SAM) for efficient fine-tuning of medical images. In the initial stage, H-SAM employs SAM's original decoder to generate a prior probabilistic mask, guiding a more intricate decoding process. Our H-SAM demonstrates a 4.78% improvement in average Dice compared to existing prompt-free SAM variants.
  arXiv  Detail & Related papers  (2024-03-27T05:55:16Z)
• UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images [47.59627416801523]
  In digital pathology, precise nuclei segmentation is pivotal yet challenged by the diversity of tissue types, staining protocols, and imaging conditions. We propose the Universal prompt-free SAM framework for Nuclei segmentation (UN-SAM) UN-SAM with exceptional performance surpasses state-of-the-arts in nuclei instance and semantic segmentation, especially the generalization capability in zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-02-26T15:35:18Z)
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation [49.827306773992376]
  Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions. Our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-12-19T15:34:52Z)
• SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation [65.52097667738884]
  We introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to integrate surgical-specific information with SAM's pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning.
  arXiv  Detail & Related papers  (2023-08-17T02:51:01Z)
• DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation [22.974876391669685]
  Segment Anything Model (SAM) shows potential for improving the cross-domain robustness of medical image segmentation. SAM performs significantly worse in automatic segmentation scenarios than when manually prompted. Decoupled SAM modifies SAM's mask decoder by introducing two new modules.
  arXiv  Detail & Related papers  (2023-06-01T09:49:11Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Domain Generalization on Medical Imaging Classification using Episodic Training with Task Augmentation [62.49837463676111]
  We propose a novel scheme of episodic training with task augmentation on medical imaging classification. Motivated by the limited number of source domains in real-world medical deployment, we consider the unique task-level overfitting.
  arXiv  Detail & Related papers  (2021-06-13T03:56:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.