Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset

• URL: http://arxiv.org/abs/2406.06039v1
• Date: Mon, 10 Jun 2024 06:17:33 GMT
• Title: Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset
• Authors: Shijie Lian, Ziyi Zhang, Hua Li, Wenjie Li, Laurence Tianruo Yang, Sam Kwong, Runmin Cong,
• Abstract summary: Underwater vision tasks often suffer from low segmentation accuracy due to the complex underwater circumstances. We construct the first large-scale underwater salient instance segmentation dataset (USIS10K) We propose an Underwater Salient Instance architecture based on Segment Anything Model (USIS-SAM) specifically for the underwater domain.
• Score: 60.14089302022989
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the breakthrough of large models, Segment Anything Model (SAM) and its extensions have been attempted to apply in diverse tasks of computer vision. Underwater salient instance segmentation is a foundational and vital step for various underwater vision tasks, which often suffer from low segmentation accuracy due to the complex underwater circumstances and the adaptive ability of models. Moreover, the lack of large-scale datasets with pixel-level salient instance annotations has impeded the development of machine learning techniques in this field. To address these issues, we construct the first large-scale underwater salient instance segmentation dataset (USIS10K), which contains 10,632 underwater images with pixel-level annotations in 7 categories from various underwater scenes. Then, we propose an Underwater Salient Instance Segmentation architecture based on Segment Anything Model (USIS-SAM) specifically for the underwater domain. We devise an Underwater Adaptive Visual Transformer (UA-ViT) encoder to incorporate underwater domain visual prompts into the segmentation network. We further design an out-of-the-box underwater Salient Feature Prompter Generator (SFPG) to automatically generate salient prompters instead of explicitly providing foreground points or boxes as prompts in SAM. Comprehensive experimental results show that our USIS-SAM method can achieve superior performance on USIS10K datasets compared to the state-of-the-art methods. Datasets and codes are released on https://github.com/LiamLian0727/USIS10K.

Related papers

• UW-SDF: Exploiting Hybrid Geometric Priors for Neural SDF Reconstruction from Underwater Multi-view Monocular Images [63.32490897641344]
  We propose a framework for reconstructing target objects from multi-view underwater images based on neural SDF. We introduce hybrid geometric priors to optimize the reconstruction process, markedly enhancing the quality and efficiency of neural SDF reconstruction.
  arXiv  Detail & Related papers  (2024-10-10T16:33:56Z)
• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• Evaluation of Segment Anything Model 2: The Role of SAM2 in the Underwater Environment [2.0554501265326794]
  The Segment Anything Model (SAM) and its extensions have been attempted for applications in various underwater visualization tasks in marine sciences. Recently, Meta has developed the Segment Anything Model 2 (SAM2), which significantly improves running speed and segmentation accuracy. This report aims to explore the potential of SAM2 in marine science by evaluating it on the underwater instance segmentation datasets benchmark UIIS and USIS10K.
  arXiv  Detail & Related papers  (2024-08-06T03:20:10Z)
• MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
  We propose a novel feature learning framework named MAS-SAM for marine animal segmentation. Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
  arXiv  Detail & Related papers  (2024-04-24T07:38:14Z)
• Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM [62.85895749882285]
  Marine Animal (MAS) involves segmenting animals within marine environments. We propose a novel feature learning framework, named Dual-SAM for high-performance MAS. Our proposed method achieves state-of-the-art performances on five widely-used MAS datasets.
  arXiv  Detail & Related papers  (2024-04-07T15:34:40Z)
• Deep Instruction Tuning for Segment Anything Model [68.7934961590075]
  Segment Anything Model (SAM) has become a research hotspot in the fields of multimedia and computer vision. SAM can support different types of segmentation prompts, but it performs much worse on text-instructed tasks. We propose two simple yet effective deep instruction tuning (DIT) methods for SAM, one is end-to-end and the other is layer-wise.
  arXiv  Detail & Related papers  (2024-03-31T11:37:43Z)
• AquaSAM: Underwater Image Foreground Segmentation [1.7482936568887284]
  This work presents AquaSAM, the first attempt to extend the success of SAM on underwater images. We develop a straightforward fine-tuning method to adapt SAM to general foreground underwater image segmentation. We demonstrate that AquaSAM outperforms the default SAM model especially at hard tasks like coral reefs.
  arXiv  Detail & Related papers  (2023-08-08T12:30:36Z)
• Transformer-based Self-Supervised Fish Segmentation in Underwater Videos [1.9249287163937976]
  We introduce a Transformer-based method that uses self-supervision for high-quality fish segmentation. We show that when trained on a set of underwater videos from one dataset, the proposed model surpasses previous CNN-based and Transformer-based self-supervised methods.
  arXiv  Detail & Related papers  (2022-06-11T01:20:48Z)
• An Underwater Image Semantic Segmentation Method Focusing on Boundaries and a Real Underwater Scene Semantic Segmentation Dataset [41.842352295729555]
  We label and establish the first underwater semantic segmentation dataset of real scene(DUT-USEG:DUT Underwater dataset) We propose a semi-supervised underwater semantic segmentation network focusing on the boundaries(US-Net: Underwater Network) Experiments show that the proposed method improves by 6.7% in three categories of holothurian, echinus, starfish in DUT-USEG dataset, and state-of-the-art results.
  arXiv  Detail & Related papers  (2021-08-26T12:05:08Z)
• SVAM: Saliency-guided Visual Attention Modeling by Autonomous Underwater Robots [16.242924916178282]
  This paper presents a holistic approach to saliency-guided visual attention modeling (SVAM) for use by autonomous underwater robots. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective salient object detection (SOD) in natural underwater images.
  arXiv  Detail & Related papers  (2020-11-12T08:17:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.