Rethinking Few-Shot Medical Image Segmentation by SAM2: A Training-Free Framework with Augmentative Prompting and Dynamic Matching

• URL: http://arxiv.org/abs/2503.04826v1
• Date: Wed, 05 Mar 2025 06:12:13 GMT
• Title: Rethinking Few-Shot Medical Image Segmentation by SAM2: A Training-Free Framework with Augmentative Prompting and Dynamic Matching
• Authors: Haiyue Zu, Jun Ge, Heting Xiao, Jile Xie, Zhangzhe Zhou, Yifan Meng, Jiayi Ni, Junjie Niu, Linlin Zhang, Li Ni, Huilin Yang,
• Abstract summary: We conceptualize 3D medical image volumes as video sequences, departing from the traditional slice-by-slice paradigm.<n>We perform extensive data augmentation on a single labeled support image and, for each frame in the query volume, algorithmically select the most analogous augmented support image.<n>We demonstrate state-of-the-art performance on benchmark few-shot medical image segmentation datasets, achieving significant improvements in accuracy and annotation efficiency.
• Score: 4.1253497486581026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The reliance on large labeled datasets presents a significant challenge in medical image segmentation. Few-shot learning offers a potential solution, but existing methods often still require substantial training data. This paper proposes a novel approach that leverages the Segment Anything Model 2 (SAM2), a vision foundation model with strong video segmentation capabilities. We conceptualize 3D medical image volumes as video sequences, departing from the traditional slice-by-slice paradigm. Our core innovation is a support-query matching strategy: we perform extensive data augmentation on a single labeled support image and, for each frame in the query volume, algorithmically select the most analogous augmented support image. This selected image, along with its corresponding mask, is used as a mask prompt, driving SAM2's video segmentation. This approach entirely avoids model retraining or parameter updates. We demonstrate state-of-the-art performance on benchmark few-shot medical image segmentation datasets, achieving significant improvements in accuracy and annotation efficiency. This plug-and-play method offers a powerful and generalizable solution for 3D medical image segmentation.

Related papers

• Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Models [17.461510586128874]
  We propose a novel method that adapts DINOv2 and Segment Anything Model 2 for retrieval-augmented few-shot medical image segmentation. Our approach uses DINOv2's feature as query to retrieve similar samples from limited annotated data, which are then encoded as memories and stored in memory bank.
  arXiv  Detail & Related papers  (2024-08-16T15:48:07Z)
• Rethinking Interactive Image Segmentation with Low Latency, High Quality, and Diverse Prompts [68.86537322287474]
  Low-latency and high-quality interactive segmentation with diverse prompts are challenging for specialist and generalist models. We propose SegNext, a next-generation interactive segmentation approach offering low latency, high quality, and diverse prompt support. Our method outperforms current state-of-the-art methods on HQSeg-44K and DAVIS, both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2024-03-31T17:02:24Z)
• Explore In-Context Segmentation via Latent Diffusion Models [132.26274147026854]
  In-context segmentation aims to segment objects using given reference images. Most existing approaches adopt metric learning or masked image modeling to build the correlation between visual prompts and input image queries. This work approaches the problem from a fresh perspective - unlocking the capability of the latent diffusion model for in-context segmentation.
  arXiv  Detail & Related papers  (2024-03-14T17:52:31Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Promise:Prompt-driven 3D Medical Image Segmentation Using Pretrained Image Foundation Models [13.08275555017179]
  We propose ProMISe, a prompt-driven 3D medical image segmentation model using only a single point prompt. We evaluate our model on two public datasets for colon and pancreas tumor segmentations.
  arXiv  Detail & Related papers  (2023-10-30T16:49:03Z)
• MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation [58.53672866662472]
  We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
  arXiv  Detail & Related papers  (2023-09-16T02:41:53Z)
• Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning [17.386754270460273]
  We present a Self-Sampling Meta SAM framework for few-shot medical image segmentation. The proposed method achieves significant improvements over state-of-the-art methods in few-shot segmentation. In conclusion, we present a novel approach for rapid online adaptation in interactive image segmentation, adapting to a new organ in just 0.83 minutes.
  arXiv  Detail & Related papers  (2023-08-31T05:20:48Z)
• Self-Prompting Large Vision Models for Few-Shot Medical Image Segmentation [14.135249795318591]
  We propose a novel perspective on self-prompting in medical vision applications. We harness the embedding space of the Segment Anything Model to prompt itself through a simple yet effective linear pixel-wise classifier. We achieve competitive results on multiple datasets.
  arXiv  Detail & Related papers  (2023-08-15T08:20:07Z)
• Few Shot Medical Image Segmentation with Cross Attention Transformer [30.54965157877615]
  We propose a novel framework for few-shot medical image segmentation, termed CAT-Net. Our proposed network mines the correlations between the support image and query image, limiting them to focus only on useful foreground information. We validated the proposed method on three public datasets: Abd-CT, Abd-MRI, and Card-MRI.
  arXiv  Detail & Related papers  (2023-03-24T09:10:14Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)
• CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-24T04:55:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.