SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentation

• URL: http://arxiv.org/abs/2310.07183v2
• Date: Wed, 20 Mar 2024 06:42:18 GMT
• Title: SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentation
• Authors: Xinrun Chen, Chengliang Wang, Haojian Ning, Shiying Li, Mei Shen,
• Abstract summary: We propose a method called SAM- OCTA for local segmentation in OCTA images. The method fine-tunes a pre-trained segment anything model (SAM) using low-rank adaptation (LoRA)
• Score: 2.452498006404167
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmenting specific targets or biomarkers is necessary to analyze optical coherence tomography angiography (OCTA) images. Previous methods typically segment all the targets in an OCTA sample, such as retinal vessels (RVs). Although these methods perform well in accuracy and precision, OCTA analyses often focusing local information within the images which has not been fulfilled. In this paper, we propose a method called SAM-OCTA for local segmentation in OCTA images. The method fine-tunes a pre-trained segment anything model (SAM) using low-rank adaptation (LoRA) and utilizes prompt points for local RVs, arteries, and veins segmentation in OCTA. To explore the effect and mechanism of prompt points, we set up global and local segmentation modes with two prompt point generation strategies, namely random selection and special annotation. Considering practical usage, we conducted extended experiments with different model scales and analyzed the model performance before and after fine-tuning besides the general segmentation task. From comprehensive experimental results with the OCTA-500 dataset, our SAM-OCTA method has achieved state-of-the-art performance in common OCTA segmentation tasks related to RV and FAZ, and it also performs accurate segmentation of artery-vein and local vessels. The code is available at https://github.com/ShellRedia/SAM-OCTA-extend.

Related papers

• SAM-OCTA2: Layer Sequence OCTA Segmentation with Fine-tuned Segment Anything Model 2 [2.314516220934268]
  Low-rank adaptation technique is adopted to fine-tune the Segment Anything Model (SAM) version 2. Method is named SAM- OCTA2 and has been experimented on the OCTA-500 dataset. It achieves state-of-the-art performance in segmenting the foveal avascular zone (FAZ) on regular 2D en-face and effectively tracks local vessels across scanning layer sequences.
  arXiv  Detail & Related papers  (2024-09-14T03:28:24Z)
• Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation [52.172885882728174]
  In medical imaging contexts, it is not uncommon for human experts to rectify segmentations of specific test samples after SAM generates its segmentation predictions. We introduce a novel approach that leverages the advantages of online machine learning to enhance Segment Anything (SA) during test time. We employ rectified annotations to perform online learning, with the aim of improving the segmentation quality of SA on medical images.
  arXiv  Detail & Related papers  (2024-06-03T03:16:25Z)
• Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero shot Medical Image Segmentation [10.444726122035133]
  We propose a simple unified framework, SaLIP, for organ segmentation. SAM is used for part based segmentation within the image, followed by CLIP to retrieve the mask corresponding to the region of interest. Finally, SAM is prompted by the retrieved ROI to segment a specific organ.
  arXiv  Detail & Related papers  (2024-04-09T14:56:34Z)
• I-MedSAM: Implicit Medical Image Segmentation with Segment Anything [24.04558900909617]
  We propose I-MedSAM, which leverages the benefits of both continuous representations and SAM to obtain better cross-domain ability and accurate boundary delineation. Our proposed method with only 1.6M trainable parameters outperforms existing methods including discrete and implicit methods.
  arXiv  Detail & Related papers  (2023-11-28T00:43:52Z)
• SAM-OCTA: A Fine-Tuning Strategy for Applying Foundation Model to OCTA Image Segmentation Tasks [2.8743451550676866]
  Low-rank adaptation technique is adopted for foundation model fine-tuning and proposed corresponding prompt point generation strategies. This method is named SAM- OCTA and has been experimented on the publicly available OCTA-500 dataset. While achieving state-of-the-art performance metrics, this method accomplishes local vessel segmentation as well as effective artery-vein segmentation.
  arXiv  Detail & Related papers  (2023-09-21T03:41:08Z)
• MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation [58.53672866662472]
  We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
  arXiv  Detail & Related papers  (2023-09-16T02:41:53Z)
• SamDSK: Combining Segment Anything Model with Domain-Specific Knowledge for Semi-Supervised Learning in Medical Image Segmentation [27.044797468878837]
  The Segment Anything Model (SAM) exhibits a capability to segment a wide array of objects in natural images. We propose a novel method that combines the SAM with domain-specific knowledge for reliable utilization of unlabeled images. Our work initiates a new direction of semi-supervised learning for medical image segmentation.
  arXiv  Detail & Related papers  (2023-08-26T04:46:10Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation [72.33139350241044]
  Weakly Supervised Semantic COCO (WSSS) based on image-level labels has attracted much attention due to low annotation costs. We propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Our SIPE achieves new state-of-the-art performance using only image-level labels.
  arXiv  Detail & Related papers  (2022-03-06T09:01:03Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.