MedLSAM: Localize and Segment Anything Model for 3D CT Images
- URL: http://arxiv.org/abs/2306.14752v3
- Date: Thu, 16 Nov 2023 07:12:46 GMT
- Title: MedLSAM: Localize and Segment Anything Model for 3D CT Images
- Authors: Wenhui Lei, Xu Wei, Xiaofan Zhang, Kang Li, Shaoting Zhang
- Abstract summary: We develop a Localize Anything Model for 3D Medical Images (MedLAM)
MedLAM is capable of directly localizing any anatomical structure using just a few template scans.
It has the potential to be seamlessly integrated with future 3D SAM models.
- Score: 14.290321536041816
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The Segment Anything Model (SAM) has recently emerged as a groundbreaking
model in the field of image segmentation. Nevertheless, both the original SAM
and its medical adaptations necessitate slice-by-slice annotations, which
directly increase the annotation workload with the size of the dataset. We
propose MedLSAM to address this issue, ensuring a constant annotation workload
irrespective of dataset size and thereby simplifying the annotation process.
Our model introduces a 3D localization foundation model capable of localizing
any target anatomical part within the body. To achieve this, we develop a
Localize Anything Model for 3D Medical Images (MedLAM), utilizing two
self-supervision tasks: unified anatomical mapping (UAM) and multi-scale
similarity (MSS) across a comprehensive dataset of 14,012 CT scans. We then
establish a methodology for accurate segmentation by integrating MedLAM with
SAM. By annotating several extreme points across three directions on a few
templates, our model can autonomously identify the target anatomical region on
all data scheduled for annotation. This allows our framework to generate a 2D
bbox for every slice of the image, which is then leveraged by SAM to carry out
segmentation. We carried out comprehensive experiments on two 3D datasets
encompassing 38 distinct organs. Our findings are twofold: 1) MedLAM is capable
of directly localizing any anatomical structure using just a few template
scans, yet its performance surpasses that of fully supervised models; 2)
MedLSAM not only aligns closely with the performance of SAM and its specialized
medical adaptations with manual prompts but achieves this with minimal reliance
on extreme point annotations across the entire dataset. Furthermore, MedLAM has
the potential to be seamlessly integrated with future 3D SAM models, paving the
way for enhanced performance.
Related papers
- Towards a Comprehensive, Efficient and Promptable Anatomic Structure Segmentation Model using 3D Whole-body CT Scans [23.573958232965104]
Segment anything model (SAM) demonstrates strong ability generalization on natural image segmentation.
For segmenting 3D radiological CT or MRI scans, a 2D SAM model has to separately handle hundreds of 2D slices.
We propose a comprehensive and scalable 3D SAM model for whole-body CT segmentation, named CT-SAM3D.
arXiv Detail & Related papers (2024-03-22T09:40:52Z) - One Model to Rule them All: Towards Universal Segmentation for Medical Images with Text Prompts [62.55349777609194]
We aim to build up a model that can Segment Anything in radiology scans, driven by Text prompts, termed as SAT.
We build up the largest and most comprehensive segmentation dataset for training, by collecting over 22K 3D medical image scans.
We have trained SAT-Nano (110M parameters) and SAT-Pro (447M parameters) demonstrating comparable performance to 72 specialist nnU-Nets trained on each dataset/subsets.
arXiv Detail & Related papers (2023-12-28T18:16:00Z) - MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image
Segmentation [58.53672866662472]
We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM.
Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments.
By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
arXiv Detail & Related papers (2023-09-16T02:41:53Z) - SAM3D: Segment Anything Model in Volumetric Medical Images [11.764867415789901]
We introduce SAM3D, an innovative adaptation tailored for 3D volumetric medical image analysis.
Unlike current SAM-based methods that segment volumetric data by converting the volume into separate 2D slices for individual analysis, our SAM3D model processes the entire 3D volume image in a unified approach.
arXiv Detail & Related papers (2023-09-07T06:05:28Z) - Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity.
We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts.
For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
arXiv Detail & Related papers (2023-07-10T17:59:40Z) - 3DSAM-adapter: Holistic Adaptation of SAM from 2D to 3D for Promptable
Medical Image Segmentation [56.50064853710202]
We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation.
Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
arXiv Detail & Related papers (2023-06-23T12:09:52Z) - Customized Segment Anything Model for Medical Image Segmentation [10.933449793055313]
We build upon the large-scale image segmentation model, Segment Anything Model (SAM), to explore the new research paradigm of customizing large-scale models for medical image segmentation.
SAMed applies the low-rank-based (LoRA) finetuning strategy to the SAM image encoder and finetunes it together with the prompt encoder and the mask decoder on labeled medical image segmentation datasets.
Our trained SAMed model achieves semantic segmentation on medical images, which is on par with the state-of-the-art methods.
arXiv Detail & Related papers (2023-04-26T19:05:34Z) - Medical SAM Adapter: Adapting Segment Anything Model for Medical Image
Segmentation [51.770805270588625]
The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation.
Recent studies and individual experiments have shown that SAM underperforms in medical image segmentation.
We propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model.
arXiv Detail & Related papers (2023-04-25T07:34:22Z) - SAMM (Segment Any Medical Model): A 3D Slicer Integration to SAM [6.172995387355581]
We introduce Segment Any Medical Model (SAMM), an extension of SAM on 3D Slicer.
SAMM achieves 0.6-second latency of a complete cycle and can infer image masks in nearly real-time.
arXiv Detail & Related papers (2023-04-12T05:39:38Z) - Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE [66.63629641650572]
We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices.
We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy.
arXiv Detail & Related papers (2020-07-09T13:23:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.