Towards Segment Anything Model (SAM) for Medical Image Segmentation: A
Survey
- URL: http://arxiv.org/abs/2305.03678v3
- Date: Fri, 11 Aug 2023 04:23:29 GMT
- Title: Towards Segment Anything Model (SAM) for Medical Image Segmentation: A
Survey
- Authors: Yichi Zhang, Rushi Jiao
- Abstract summary: We discuss efforts to extend the success of the Segment Anything Model to medical image segmentation tasks.
Many insights are drawn to guide future research to develop foundation models for medical image analysis.
- Score: 8.76496233192512
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Due to the flexibility of prompting, foundation models have become the
dominant force in the domains of natural language processing and image
generation. With the recent introduction of the Segment Anything Model (SAM),
the prompt-driven paradigm has entered the realm of image segmentation,
bringing with a range of previously unexplored capabilities. However, it
remains unclear whether it can be applicable to medical image segmentation due
to the significant differences between natural images and medical images.In
this work, we summarize recent efforts to extend the success of SAM to medical
image segmentation tasks, including both empirical benchmarking and
methodological adaptations, and discuss potential future directions for SAM in
medical image segmentation. Although directly applying SAM to medical image
segmentation cannot obtain satisfying performance on multi-modal and
multi-target medical datasets, many insights are drawn to guide future research
to develop foundation models for medical image analysis. To facilitate future
research, we maintain an active repository that contains up-to-date paper list
and open-source project summary at https://github.com/YichiZhang98/SAM4MIS.
Related papers
- Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation [52.172885882728174]
In medical imaging contexts, it is not uncommon for human experts to rectify segmentations of specific test samples after SAM generates its segmentation predictions.
We introduce a novel approach that leverages the advantages of online machine learning to enhance Segment Anything (SA) during test time.
We employ rectified annotations to perform online learning, with the aim of improving the segmentation quality of SA on medical images.
arXiv Detail & Related papers (2024-06-03T03:16:25Z) - Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
We introduce the Med-ST framework for fine-grained spatial and temporal modeling.
For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views.
For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
arXiv Detail & Related papers (2024-05-30T03:15:09Z) - Segment Any Medical Model Extended [39.80956010574076]
We introduce SAMM Extended (SAMME), a platform that integrates new SAM variant models, adopts faster communication protocols, accommodates new interactive modes, and allows for fine-tuning of subcomponents of the models.
These features can expand the potential of foundation models like SAM, and the results can be translated to applications such as image-guided therapy, mixed reality interaction, robotic navigation, and data augmentation.
arXiv Detail & Related papers (2024-03-26T21:37:25Z) - Segment Anything Model for Medical Image Segmentation: Current
Applications and Future Directions [8.216028136706948]
The recent introduction of the Segment Anything Model (SAM) signifies a noteworthy expansion of the prompt-driven paradigm into the domain of image segmentation.
We provide a comprehensive overview of recent endeavors aimed at extending the efficacy of SAM to medical image segmentation tasks.
We explore potential avenues for future research directions in SAM's role within medical image segmentation.
arXiv Detail & Related papers (2024-01-07T14:25:42Z) - C^2M-DoT: Cross-modal consistent multi-view medical report generation
with domain transfer network [67.97926983664676]
We propose a cross-modal consistent multi-view medical report generation with a domain transfer network (C2M-DoT)
C2M-DoT substantially outperforms state-of-the-art baselines in all metrics.
arXiv Detail & Related papers (2023-10-09T02:31:36Z) - MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image
Segmentation [58.53672866662472]
We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM.
Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments.
By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
arXiv Detail & Related papers (2023-09-16T02:41:53Z) - Empirical Analysis of a Segmentation Foundation Model in Prostate
Imaging [9.99042549094606]
We consider a recently developed foundation model for medical image segmentation, UniverSeg.
We conduct an empirical evaluation study in the context of prostate imaging and compare it against the conventional approach of training a task-specific segmentation model.
arXiv Detail & Related papers (2023-07-06T20:00:52Z) - Generalist Vision Foundation Models for Medical Imaging: A Case Study of
Segment Anything Model on Zero-Shot Medical Segmentation [5.547422331445511]
We report quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks.
Our study indicates the versatility of generalist vision foundation models on medical imaging.
arXiv Detail & Related papers (2023-04-25T08:07:59Z) - Medical SAM Adapter: Adapting Segment Anything Model for Medical Image
Segmentation [51.770805270588625]
The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation.
Recent studies and individual experiments have shown that SAM underperforms in medical image segmentation.
We propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model.
arXiv Detail & Related papers (2023-04-25T07:34:22Z) - SAM.MD: Zero-shot medical image segmentation capabilities of the Segment
Anything Model [1.1221592576472588]
We evaluate the zero-shot capabilities of the Segment Anything Model for medical image segmentation.
We show that SAM generalizes well to CT data, making it a potential catalyst for the advancement of semi-automatic segmentation tools.
arXiv Detail & Related papers (2023-04-10T18:20:29Z) - MedSegDiff-V2: Diffusion based Medical Image Segmentation with
Transformer [53.575573940055335]
We propose a novel Transformer-based Diffusion framework, called MedSegDiff-V2.
We verify its effectiveness on 20 medical image segmentation tasks with different image modalities.
arXiv Detail & Related papers (2023-01-19T03:42:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.