Compositional Oil Spill Detection Based on Object Detector and Adapted Segment Anything Model from SAR Images

• URL: http://arxiv.org/abs/2401.07502v2
• Date: Sun, 22 Dec 2024 06:35:53 GMT
• Title: Compositional Oil Spill Detection Based on Object Detector and Adapted Segment Anything Model from SAR Images
• Authors: Wenhui Wu, Man Sing Wong, Xinyu Yu, Guoqiang Shi, Coco Yin Tung Kwok, Kang Zou,
• Abstract summary: We propose a composite oil spill detection framework, SAM-OIL, comprising an object detector (e.g., YOLOv8), an Adapted Segment Anything Model (SAM) and an Ordered Mask Fusion (OMF) module.<n> SAM-OIL is the first application of the powerful SAM in oil spill detection. Specifically, the SAM-OIL strategy uses YOLOv8 to obtain the categories and bounding boxes of oil spill-related objects, then inputs bounding boxes into the Adapted SAM to retrieve category-agnostic masks, and finally adopts the OMF module to fuse the masks and categories.
• Score: 3.2843040151689586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation-based methods have attracted extensive attention in oil spill detection from SAR images. However, the existing approaches require a large number of finely annotated segmentation samples in the training stage. To alleviate this issue, we propose a composite oil spill detection framework, SAM-OIL, comprising an object detector (e.g., YOLOv8), an Adapted Segment Anything Model (SAM), and an Ordered Mask Fusion (OMF) module. SAM-OIL is the first application of the powerful SAM in oil spill detection. Specifically, the SAM-OIL strategy uses YOLOv8 to obtain the categories and bounding boxes of oil spill-related objects, then inputs bounding boxes into the Adapted SAM to retrieve category-agnostic masks, and finally adopts the OMF module to fuse the masks and categories. The Adapted SAM, combining a frozen SAM with a learnable Adapter module, can enhance SAM's ability to segment ambiguous objects. The OMF module, a parameter-free method, can effectively resolve pixel category conflicts within SAM. Experimental results demonstrate that SAM-OIL surpasses existing semantic segmentation-based oil spill detection methods, achieving mIoU of 69.52\%. The results also indicated that both OMF and Adapter modules can effectively improve the accuracy in SAM-OIL.

Related papers

• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM [25.136857576951282]
  Mask-injected framework SAM-MI integrates SAM with OVSS models to address challenges.<n> SAM-MI employs a Text-guided Sparse Point Prompter to sample sparse prompts for SAM instead of previous dense grid-like prompts.<n>DMI incorporates SAM-generated masks for guidance at low-frequency and high-frequency separately, rather than directly combining them with labels.
  arXiv  Detail & Related papers  (2025-11-25T07:52:07Z)
• SAMRefiner: Taming Segment Anything Model for Universal Mask Refinement [40.37217744643069]
  We propose a universal and efficient approach by adapting SAM to the mask refinement task. Specifically, we introduce a multi-prompt excavation strategy to mine diverse input prompts for SAM. We extend our method to SAMRefiner++ by introducing an additional IoU adaption step to further boost the performance of the generic SAMRefiner on the target dataset.
  arXiv  Detail & Related papers  (2025-02-10T18:33:15Z)
• Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation [47.789013598970925]
  We propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Our model outperforms the state-of-the-art semi-supervised segmentation approaches.
  arXiv  Detail & Related papers  (2024-12-18T11:19:23Z)
• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• Tuning a SAM-Based Model with Multi-Cognitive Visual Adapter to Remote Sensing Instance Segmentation [4.6570959687411975]
  The Segment Anything Model (SAM) demonstrates exceptional generalization capabilities. SAM's lack of pretraining on massive remote sensing images and its interactive structure limit its automatic mask prediction capabilities. A Multi- cognitive SAM-Based Instance Model (MC-SAM SEG) is introduced to employ SAM on remote sensing domain. The proposed method named MC-SAM SEG extracts high-quality features by fine-tuning the SAM-Mona encoder along with a feature aggregator.
  arXiv  Detail & Related papers  (2024-08-16T07:23:22Z)
• Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection [58.241593208031816]
  Segment Anything Model (SAM) has been proposed as a visual fundamental model, which gives strong segmentation and generalization capabilities. We propose a Multi-scale and Detail-enhanced SAM (MDSAM) for Salient Object Detection (SOD) Experimental results demonstrate the superior performance of our model on multiple SOD datasets.
  arXiv  Detail & Related papers  (2024-08-08T09:09:37Z)
• MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
  We propose a novel feature learning framework named MAS-SAM for marine animal segmentation. Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
  arXiv  Detail & Related papers  (2024-04-24T07:38:14Z)
• Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM [62.85895749882285]
  Marine Animal (MAS) involves segmenting animals within marine environments. We propose a novel feature learning framework, named Dual-SAM for high-performance MAS. Our proposed method achieves state-of-the-art performances on five widely-used MAS datasets.
  arXiv  Detail & Related papers  (2024-04-07T15:34:40Z)
• PosSAM: Panoptic Open-vocabulary Segment Anything [58.72494640363136]
  PosSAM is an open-vocabulary panoptic segmentation model that unifies the strengths of the Segment Anything Model (SAM) with the vision-native CLIP model in an end-to-end framework. We introduce a Mask-Aware Selective Ensembling (MASE) algorithm that adaptively enhances the quality of generated masks and boosts the performance of open-vocabulary classification during inference for each image.
  arXiv  Detail & Related papers  (2024-03-14T17:55:03Z)
• WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images [8.179859593451285]
  We present WSI-SAM, enhancing Segment Anything Model (SAM) with precise object segmentation capabilities for histopathology images. To fully exploit pretrained knowledge while minimizing training overhead, we keep SAM frozen, introducing only minimal extra parameters. Our model outperforms SAM by 4.1 and 2.5 percent points on a ductal carcinoma in situ (DCIS) segmentation tasks and breast cancer metastasis segmentation task.
  arXiv  Detail & Related papers  (2024-03-14T10:30:43Z)
• VRP-SAM: SAM with Visual Reference Prompt [73.05676082695459]
  We propose a novel Visual Reference Prompt (VRP) encoder that empowers the Segment Anything Model (SAM) to utilize annotated reference images as prompts for segmentation. In essence, VRP-SAM can utilize annotated reference images to comprehend specific objects and perform segmentation of specific objects in target image.
  arXiv  Detail & Related papers  (2024-02-27T17:58:09Z)
• PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation [19.65118388712439]
  We introduce a novel prompt-driven adapter into SAM, namely Prompt Adapter Segment Anything Model (PA-SAM) By exclusively training the prompt adapter, PA-SAM extracts detailed information from images and optimize the mask decoder feature at both sparse and dense prompt levels. Experimental results demonstrate that our PA-SAM outperforms other SAM-based methods in high-quality, zero-shot, and open-set segmentation.
  arXiv  Detail & Related papers  (2024-01-23T19:20:22Z)
• Boosting Segment Anything Model Towards Open-Vocabulary Learning [69.24734826209367]
  Segment Anything Model (SAM) has emerged as a new paradigmatic vision foundation model. Despite SAM finding applications and adaptations in various domains, its primary limitation lies in the inability to grasp object semantics. We present Sambor to seamlessly integrate SAM with the open-vocabulary object detector in an end-to-end framework.
  arXiv  Detail & Related papers  (2023-12-06T17:19:00Z)
• Repurposing SAM for User-Defined Semantics Aware Segmentation [23.88643687043431]
  We propose U-SAM, a novel framework that imbibes semantic awareness into SAM. U-SAM provides pixel-level semantic annotations for images without requiring any labeled/unlabeled samples from the test data distribution. We evaluate U-SAM on PASCAL VOC 2012 and MSCOCO-80, achieving significant mIoU improvements of +17.95% and +520%, respectively.
  arXiv  Detail & Related papers  (2023-12-05T01:37:18Z)
• DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation [22.974876391669685]
  Segment Anything Model (SAM) shows potential for improving the cross-domain robustness of medical image segmentation. SAM performs significantly worse in automatic segmentation scenarios than when manually prompted. Decoupled SAM modifies SAM's mask decoder by introducing two new modules.
  arXiv  Detail & Related papers  (2023-06-01T09:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.