Are foundation models efficient for medical image segmentation?

• URL: http://arxiv.org/abs/2311.04847v1
• Date: Wed, 8 Nov 2023 17:33:09 GMT
• Title: Are foundation models efficient for medical image segmentation?
• Abstract summary: The Segment Anything model (SAM) asserts an ability to segment a wide spectrum of objects but required supervised training at unprecedented scale. We compared SAM's performance (against clinical ground truth) and resources (labeling time, compute) to a modality-specific, label-free self-supervised learning (SSL) method on 25 measurements for 100 cardiac ultrasounds.
• Score: 0.8158530638728501
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Foundation models are experiencing a surge in popularity. The Segment Anything model (SAM) asserts an ability to segment a wide spectrum of objects but required supervised training at unprecedented scale. We compared SAM's performance (against clinical ground truth) and resources (labeling time, compute) to a modality-specific, label-free self-supervised learning (SSL) method on 25 measurements for 100 cardiac ultrasounds. SAM performed poorly and required significantly more labeling and computing resources, demonstrating worse efficiency than SSL.

Related papers

• From Specialist to Generalist: Unlocking SAM's Learning Potential on Unlabeled Medical Images [12.062960289184199]
  We introduce SC-SAM, a specialist-generalist framework where U-Net provides point-based prompts and pseudo-labels to guide SAM's adaptation.<n>This reciprocal guidance forms a bidirectional co-training loop that allows both models to effectively exploit the unlabeled data.<n>Our method achieves state-of-the-art results, outperforming other existing semi-supervised SAM variants and even medical foundation models like MedSAM.
  arXiv  Detail & Related papers  (2026-01-25T18:13:48Z)
• SSL-MedSAM2: A Semi-supervised Medical Image Segmentation Framework Powered by Few-shot Learning of SAM2 [4.5218971269045225]
  Semi-supervised learning has emerged as an appealing strategy in training with limited annotations.<n>We propose a novel SSL framework SSL-MedSAM2, which contains a training-free few-shot learning branch TFFS-MedSAM2 for pseudo label generation.<n>The results on MICCAI2025 challenge demonstrate an outstanding performance of SSL-MedSAM2 among other methods.
  arXiv  Detail & Related papers  (2025-12-12T13:33:38Z)
• Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classification [113.6840565194525]
  Real-world datasets often follow a long-tailed distribution, making generalization to tail classes difficult.<n>Recent methods resorted to long-tail variants of Sharpness-Aware Minimization (SAM) to improve generalization by flattening the loss landscape.<n>We introduce Focal-SAM, which assigns different penalties to class-wise, achieving fine-grained control without extra backpropagations.
  arXiv  Detail & Related papers  (2025-05-03T03:01:28Z)
• Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation [47.789013598970925]
  We propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Our model outperforms the state-of-the-art semi-supervised segmentation approaches.
  arXiv  Detail & Related papers  (2024-12-18T11:19:23Z)
• Annotation-Efficient Task Guidance for Medical Segment Anything [0.31077024712075796]
  Medical image segmentation is a key task in the imaging workflow, influencing many image-based decisions. Traditional, fully-supervised segmentation models rely on large amounts of labeled training data, which can be an expensive, time-consuming, and error-prone process. We propose SAM-Mix, a novel multitask learning framework for medical image segmentation.
  arXiv  Detail & Related papers  (2024-12-11T17:47:00Z)
• MCP-MedSAM: A Powerful Lightweight Medical Segment Anything Model Trained with a Single GPU in Just One Day [0.6827423171182151]
  Medical image segmentation involves partitioning medical images into meaningful regions, with a focus on identifying anatomical structures or abnormalities. The Anything Model (SAM) has prompted researchers to adapt it for the medical domain to improve performance across various tasks. We propose MCP-MedSAM, a powerful and lightweight medical SAM model designed to be trainable on a single GPU within one day.
  arXiv  Detail & Related papers  (2024-12-08T10:50:59Z)
• SAM-Driven Weakly Supervised Nodule Segmentation with Uncertainty-Aware Cross Teaching [13.5553526185399]
  Automated nodule segmentation is essential for computer-assisted diagnosis in ultrasound images. Recently, segmentation foundation models like SAM have shown impressive generalizability on natural images. In this work, we devise a novel weakly supervised framework that effectively utilizes the segmentation foundation model to generate pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-18T14:27:54Z)
• Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation [52.172885882728174]
  In medical imaging contexts, it is not uncommon for human experts to rectify segmentations of specific test samples after SAM generates its segmentation predictions. We introduce a novel approach that leverages the advantages of online machine learning to enhance Segment Anything (SA) during test time. We employ rectified annotations to perform online learning, with the aim of improving the segmentation quality of SA on medical images.
  arXiv  Detail & Related papers  (2024-06-03T03:16:25Z)
• TinySAM: Pushing the Envelope for Efficient Segment Anything Model [76.21007576954035]
  We propose a framework to obtain a tiny segment anything model (TinySAM) while maintaining the strong zero-shot performance. We first propose a full-stage knowledge distillation method with hard prompt sampling and hard mask weighting strategy to distill a lightweight student model. We also adapt the post-training quantization to the promptable segmentation task and further reduce the computational cost.
  arXiv  Detail & Related papers  (2023-12-21T12:26:11Z)
• ASLseg: Adapting SAM in the Loop for Semi-supervised Liver Tumor Segmentation [2.3617131367115705]
  Liver tumor segmentation is essential for computer-aided diagnosis, surgical planning, and prognosis evaluation. Semi-Supervised Learning (SSL) is a common technique to address these challenges. We propose a novel semi-supervised framework, named ASLseg, which can effectively adapt the SAM to the SSL setting.
  arXiv  Detail & Related papers  (2023-12-13T08:31:26Z)
• nnSAM: Plug-and-play Segment Anything Model Improves nnUNet Performance [12.169801149021566]
  The Segment Anything Model (SAM) has emerged as a versatile tool for image segmentation without specific domain training. Traditional models like nnUNet perform automatic segmentation during inference but need extensive domain-specific training. We propose nnSAM, integrating SAM's robust feature extraction with nnUNet's automatic configuration to enhance segmentation accuracy on small datasets.
  arXiv  Detail & Related papers  (2023-09-29T04:26:25Z)
• MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation [58.53672866662472]
  We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
  arXiv  Detail & Related papers  (2023-09-16T02:41:53Z)
• SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation [65.52097667738884]
  We introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to integrate surgical-specific information with SAM's pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning.
  arXiv  Detail & Related papers  (2023-08-17T02:51:01Z)
• How to Efficiently Adapt Large Segmentation Model(SAM) to Medical Images [15.181219203629643]
  Segment Anything (SAM) exhibits impressive capabilities in zero-shot segmentation for natural images. However, when applied to medical images, SAM suffers from noticeable performance drop. In this work, we propose to freeze SAM encoder and finetune a lightweight task-specific prediction head.
  arXiv  Detail & Related papers  (2023-06-23T18:34:30Z)
• Ladder Fine-tuning approach for SAM integrating complementary network [5.46706034286531]
  In medical imaging, the lack of training samples due to privacy concerns and other factors presents a major challenge for applying these generalized models to medical image segmentation task. In this study, we propose to combine a complementary Convolutional Neural Network (CNN) along with the standard SAM network for medical image segmentation. This strategy significantly reduces trainnig time and achieves competitive results on publicly available dataset.
  arXiv  Detail & Related papers  (2023-06-22T08:36:17Z)
• Learnable Weight Initialization for Volumetric Medical Image Segmentation [66.3030435676252]
  We propose a learnable weight-based hybrid medical image segmentation approach. Our approach is easy to integrate into any hybrid model and requires no external training data. Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-15T17:55:05Z)
• DATA: Domain-Aware and Task-Aware Pre-training [94.62676913928831]
  We present DATA, a simple yet effective NAS approach specialized for self-supervised learning (SSL) Our method achieves promising results across a wide range of computation costs on downstream tasks, including image classification, object detection and semantic segmentation.
  arXiv  Detail & Related papers  (2022-03-17T02:38:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.