Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset

• URL: http://arxiv.org/abs/2406.06039v1
• Date: Mon, 10 Jun 2024 06:17:33 GMT
• Title: Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset
• Authors: Shijie Lian, Ziyi Zhang, Hua Li, Wenjie Li, Laurence Tianruo Yang, Sam Kwong, Runmin Cong,
• Abstract summary: Underwater vision tasks often suffer from low segmentation accuracy due to the complex underwater circumstances. We construct the first large-scale underwater salient instance segmentation dataset (USIS10K) We propose an Underwater Salient Instance architecture based on Segment Anything Model (USIS-SAM) specifically for the underwater domain.
• Score: 60.14089302022989
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the breakthrough of large models, Segment Anything Model (SAM) and its extensions have been attempted to apply in diverse tasks of computer vision. Underwater salient instance segmentation is a foundational and vital step for various underwater vision tasks, which often suffer from low segmentation accuracy due to the complex underwater circumstances and the adaptive ability of models. Moreover, the lack of large-scale datasets with pixel-level salient instance annotations has impeded the development of machine learning techniques in this field. To address these issues, we construct the first large-scale underwater salient instance segmentation dataset (USIS10K), which contains 10,632 underwater images with pixel-level annotations in 7 categories from various underwater scenes. Then, we propose an Underwater Salient Instance Segmentation architecture based on Segment Anything Model (USIS-SAM) specifically for the underwater domain. We devise an Underwater Adaptive Visual Transformer (UA-ViT) encoder to incorporate underwater domain visual prompts into the segmentation network. We further design an out-of-the-box underwater Salient Feature Prompter Generator (SFPG) to automatically generate salient prompters instead of explicitly providing foreground points or boxes as prompts in SAM. Comprehensive experimental results show that our USIS-SAM method can achieve superior performance on USIS10K datasets compared to the state-of-the-art methods. Datasets and codes are released on https://github.com/LiamLian0727/USIS10K.

Related papers

• MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
  We propose a novel feature learning framework named MAS-SAM for marine animal segmentation. Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
  arXiv  Detail & Related papers  (2024-04-24T07:38:14Z)
• Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM [62.85895749882285]
  Marine Animal (MAS) involves segmenting animals within marine environments. We propose a novel feature learning framework, named Dual-SAM for high-performance MAS. Our proposed method achieves state-of-the-art performances on five widely-used MAS datasets.
  arXiv  Detail & Related papers  (2024-04-07T15:34:40Z)
• Deep Instruction Tuning for Segment Anything Model [68.7934961590075]
  Segment Anything Model (SAM) has become a research hotspot in the fields of multimedia and computer vision. SAM can support different types of segmentation prompts, but it performs much worse on text-instructed tasks. We propose two simple yet effective deep instruction tuning (DIT) methods for SAM, one is end-to-end and the other is layer-wise.
  arXiv  Detail & Related papers  (2024-03-31T11:37:43Z)
• AquaSAM: Underwater Image Foreground Segmentation [1.7482936568887284]
  This work presents AquaSAM, the first attempt to extend the success of SAM on underwater images. We develop a straightforward fine-tuning method to adapt SAM to general foreground underwater image segmentation. We demonstrate that AquaSAM outperforms the default SAM model especially at hard tasks like coral reefs.
  arXiv  Detail & Related papers  (2023-08-08T12:30:36Z)
• SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model [85.85899655118087]
  We develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS. SAMRS totally possesses 105,090 images and 1,668,241 instances, surpassing existing high-resolution RS segmentation datasets in size by several orders of magnitude.
  arXiv  Detail & Related papers  (2023-05-03T10:58:07Z)
• Semantic-aware Texture-Structure Feature Collaboration for Underwater Image Enhancement [58.075720488942125]
  Underwater image enhancement has become an attractive topic as a significant technology in marine engineering and aquatic robotics. We develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model. We also apply the proposed algorithm to the underwater salient object detection task to reveal the favorable semantic-aware ability for high-level vision tasks.
  arXiv  Detail & Related papers  (2022-11-19T07:50:34Z)
• Transformer-based Self-Supervised Fish Segmentation in Underwater Videos [1.9249287163937976]
  We introduce a Transformer-based method that uses self-supervision for high-quality fish segmentation. We show that when trained on a set of underwater videos from one dataset, the proposed model surpasses previous CNN-based and Transformer-based self-supervised methods.
  arXiv  Detail & Related papers  (2022-06-11T01:20:48Z)
• An Underwater Image Semantic Segmentation Method Focusing on Boundaries and a Real Underwater Scene Semantic Segmentation Dataset [41.842352295729555]
  We label and establish the first underwater semantic segmentation dataset of real scene(DUT-USEG:DUT Underwater dataset) We propose a semi-supervised underwater semantic segmentation network focusing on the boundaries(US-Net: Underwater Network) Experiments show that the proposed method improves by 6.7% in three categories of holothurian, echinus, starfish in DUT-USEG dataset, and state-of-the-art results.
  arXiv  Detail & Related papers  (2021-08-26T12:05:08Z)
• SVAM: Saliency-guided Visual Attention Modeling by Autonomous Underwater Robots [16.242924916178282]
  This paper presents a holistic approach to saliency-guided visual attention modeling (SVAM) for use by autonomous underwater robots. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective salient object detection (SOD) in natural underwater images.
  arXiv  Detail & Related papers  (2020-11-12T08:17:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.