Self-Prompting Polyp Segmentation in Colonoscopy using Hybrid Yolo-SAM 2 Model

• URL: http://arxiv.org/abs/2409.09484v1
• Date: Sat, 14 Sep 2024 17:11:37 GMT
• Title: Self-Prompting Polyp Segmentation in Colonoscopy using Hybrid Yolo-SAM 2 Model
• Authors: Mobina Mansoori, Sajjad Shahabodini, Jamshid Abouei, Konstantinos N. Plataniotis, Arash Mohammadi,
• Abstract summary: This paper presents a novel approach to polyp segmentation by integrating the Segment Anything Model (SAM 2) with the YOLOv8 model. Our method leverages YOLOv8's bounding box predictions to autonomously generate input prompts for SAM 2, thereby reducing the need for manual annotations. We conducted exhaustive tests on five benchmark colonoscopy image datasets and two colonoscopy video datasets, demonstrating that our method exceeds state-of-the-art models in both image and video segmentation tasks.
• Score: 18.61909523131399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Early diagnosis and treatment of polyps during colonoscopy are essential for reducing the incidence and mortality of Colorectal Cancer (CRC). However, the variability in polyp characteristics and the presence of artifacts in colonoscopy images and videos pose significant challenges for accurate and efficient polyp detection and segmentation. This paper presents a novel approach to polyp segmentation by integrating the Segment Anything Model (SAM 2) with the YOLOv8 model. Our method leverages YOLOv8's bounding box predictions to autonomously generate input prompts for SAM 2, thereby reducing the need for manual annotations. We conducted exhaustive tests on five benchmark colonoscopy image datasets and two colonoscopy video datasets, demonstrating that our method exceeds state-of-the-art models in both image and video segmentation tasks. Notably, our approach achieves high segmentation accuracy using only bounding box annotations, significantly reducing annotation time and effort. This advancement holds promise for enhancing the efficiency and scalability of polyp detection in clinical settings https://github.com/sajjad-sh33/YOLO_SAM2.

Related papers

• Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer Detection [18.61909523131399]
  Polyp segmentation plays a crucial role in the early detection and diagnosis of colorectal cancer. Recently, Meta AI Research released a general Segment Anything Model 2 (SAM 2), which has demonstrated promising performance in several segmentation tasks. In this manuscript, we evaluate the performance of SAM 2 in segmenting polyps under various prompted settings.
  arXiv  Detail & Related papers  (2024-08-12T02:10:18Z)
• ASPS: Augmented Segment Anything Model for Polyp Segmentation [77.25557224490075]
  The Segment Anything Model (SAM) has introduced unprecedented potential for polyp segmentation. SAM's Transformer-based structure prioritizes global and low-frequency information. CFA integrates a trainable CNN encoder branch with a frozen ViT encoder, enabling the integration of domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-06-30T14:55:32Z)
• ECC-PolypDet: Enhanced CenterNet with Contrastive Learning for Automatic Polyp Detection [88.4359020192429]
  Existing methods either involve computationally expensive context aggregation or lack prior modeling of polyps, resulting in poor performance in challenging cases. In this paper, we propose the Enhanced CenterNet with Contrastive Learning (ECC-PolypDet), a two-stage training & end-to-end inference framework. Box-assisted Contrastive Learning (BCL) during training to minimize the intra-class difference and maximize the inter-class difference between foreground polyps and backgrounds, enabling our model to capture concealed polyps. In the fine-tuning stage, we introduce the IoU-guided Sample Re-weighting
  arXiv  Detail & Related papers  (2024-01-10T07:03:41Z)
• Polyp-SAM: Transfer SAM for Polyp Segmentation [2.4492242722754107]
  Segment Anything Model (SAM) has recently gained much attention in both natural and medical image segmentation. We propose Poly-SAM, a finetuned SAM model for polyp segmentation, and compare its performance to several state-of-the-art polyp segmentation models. Our Polyp-SAM achieves state-of-the-art performance on two datasets and impressive performance on three datasets, with dice scores all above 88%.
  arXiv  Detail & Related papers  (2023-04-29T16:11:06Z)
• Lesion-aware Dynamic Kernel for Polyp Segmentation [49.63274623103663]
  We propose a lesion-aware dynamic network (LDNet) for polyp segmentation. It is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. This simple but effective scheme endows our model with powerful segmentation performance and generalization capability.
  arXiv  Detail & Related papers  (2023-01-12T09:53:57Z)
• BoxPolyp:Boost Generalized Polyp Segmentation Using Extra Coarse Bounding Box Annotations [79.17754846553866]
  We propose a boosted BoxPolyp model to make full use of both accurate mask and extra coarse box annotations. In practice, box annotations are applied to alleviate the over-fitting issue of previous polyp segmentation models. Our proposed model outperforms previous state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-12-07T07:45:50Z)
• Stepwise Feature Fusion: Local Guides Global [14.394421688712052]
  We propose a new State-Of-The-Art model for medical image segmentation, the SSFormer, which uses a pyramid Transformer encoder to improve the generalization ability of models. Our proposed Progressive Locality Decoder can be adapted to the pyramid Transformer backbone to emphasize local features and attention dispersion.
  arXiv  Detail & Related papers  (2022-03-07T10:36:38Z)
• Advances in Artificial Intelligence to Reduce Polyp Miss Rates during Colonoscopy [0.7619404259039283]
  We introduce a new deep neural network architecture, which achieves state-of-the-art performance for polyp segmentation. Our algorithm could be integrated into colonoscopy practice and assist gastroenterologists by reducing the number of polyps missed.
  arXiv  Detail & Related papers  (2021-05-16T16:10:32Z)
• AG-CUResNeSt: A Novel Method for Colon Polyp Segmentation [0.0]
  This paper proposes a novel neural network architecture called AG-CUResNeSt, which enhances Coupled UNets using the robust ResNeSt backbone and attention gates. We show that our proposed method achieves state-of-the-art accuracy compared to existing methods.
  arXiv  Detail & Related papers  (2021-05-02T06:36:36Z)
• Colonoscopy Polyp Detection: Domain Adaptation From Medical Report Images to Real-time Videos [76.37907640271806]
  We propose an Image-video-joint polyp detection network (Ivy-Net) to address the domain gap between colonoscopy images from historical medical reports and real-time videos. Experiments on the collected dataset demonstrate that our Ivy-Net achieves the state-of-the-art result on colonoscopy video.
  arXiv  Detail & Related papers  (2020-12-31T10:33:09Z)
• PraNet: Parallel Reverse Attention Network for Polyp Segmentation [155.93344756264824]
  We propose a parallel reverse attention network (PraNet) for accurate polyp segmentation in colonoscopy images. We first aggregate the features in high-level layers using a parallel partial decoder (PPD) In addition, we mine the boundary cues using a reverse attention (RA) module, which is able to establish the relationship between areas and boundary cues.
  arXiv  Detail & Related papers  (2020-06-13T08:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.