Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection

• URL: http://arxiv.org/abs/2408.04326v1
• Date: Thu, 8 Aug 2024 09:09:37 GMT
• Title: Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection
• Authors: Shixuan Gao, Pingping Zhang, Tianyu Yan, Huchuan Lu,
• Abstract summary: Segment Anything Model (SAM) has been proposed as a visual fundamental model, which gives strong segmentation and generalization capabilities. We propose a Multi-scale and Detail-enhanced SAM (MDSAM) for Salient Object Detection (SOD) Experimental results demonstrate the superior performance of our model on multiple SOD datasets.
• Score: 58.241593208031816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Salient Object Detection (SOD) aims to identify and segment the most prominent objects in images. Advanced SOD methods often utilize various Convolutional Neural Networks (CNN) or Transformers for deep feature extraction. However, these methods still deliver low performance and poor generalization in complex cases. Recently, Segment Anything Model (SAM) has been proposed as a visual fundamental model, which gives strong segmentation and generalization capabilities. Nonetheless, SAM requires accurate prompts of target objects, which are unavailable in SOD. Additionally, SAM lacks the utilization of multi-scale and multi-level information, as well as the incorporation of fine-grained details. To address these shortcomings, we propose a Multi-scale and Detail-enhanced SAM (MDSAM) for SOD. Specifically, we first introduce a Lightweight Multi-Scale Adapter (LMSA), which allows SAM to learn multi-scale information with very few trainable parameters. Then, we propose a Multi-Level Fusion Module (MLFM) to comprehensively utilize the multi-level information from the SAM's encoder. Finally, we propose a Detail Enhancement Module (DEM) to incorporate SAM with fine-grained details. Experimental results demonstrate the superior performance of our model on multiple SOD datasets and its strong generalization on other segmentation tasks. The source code is released at https://github.com/BellyBeauty/MDSAM.

Related papers

• Adapting Segment Anything Model to Multi-modal Salient Object Detection with Semantic Feature Fusion Guidance [15.435695491233982]
  We propose a novel framework to explore and exploit the powerful feature representation and zero-shot generalization ability of the Segment Anything Model (SAM) for multi-modal salient object detection (SOD) We develop underlineSAM with seunderlinemantic funderlineeature fuunderlinesion guidancunderlinee (Sammese) In the image encoder, a multi-modal adapter is proposed to adapt the single-modal SAM to multi-modal information. Specifically, in the mask decoder, a semantic-geometric
  arXiv  Detail & Related papers  (2024-08-27T13:47:31Z)
• Tuning a SAM-Based Model with Multi-Cognitive Visual Adapter to Remote Sensing Instance Segmentation [4.6570959687411975]
  The Segment Anything Model (SAM) demonstrates exceptional generalization capabilities. SAM's lack of pretraining on massive remote sensing images and its interactive structure limit its automatic mask prediction capabilities. A Multi- cognitive SAM-Based Instance Model (MC-SAM SEG) is introduced to employ SAM on remote sensing domain. The proposed method named MC-SAM SEG extracts high-quality features by fine-tuning the SAM-Mona encoder along with a feature aggregator.
  arXiv  Detail & Related papers  (2024-08-16T07:23:22Z)
• MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
  We propose a novel feature learning framework named MAS-SAM for marine animal segmentation. Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
  arXiv  Detail & Related papers  (2024-04-24T07:38:14Z)
• Moving Object Segmentation: All You Need Is SAM (and Flow) [82.78026782967959]
  We investigate two models for combining SAM with optical flow that harness the segmentation power of SAM with the ability of flow to discover and group moving objects. In the first model, we adapt SAM to take optical flow, rather than RGB, as an input. In the second, SAM takes RGB as an input, and flow is used as a segmentation prompt. These surprisingly simple methods, without any further modifications, outperform all previous approaches by a considerable margin in both single and multi-object benchmarks.
  arXiv  Detail & Related papers  (2024-04-18T17:59:53Z)
• Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM [62.85895749882285]
  Marine Animal (MAS) involves segmenting animals within marine environments. We propose a novel feature learning framework, named Dual-SAM for high-performance MAS. Our proposed method achieves state-of-the-art performances on five widely-used MAS datasets.
  arXiv  Detail & Related papers  (2024-04-07T15:34:40Z)
• Task-Aware Low-Rank Adaptation of Segment Anything Model [4.5963832382272125]
  The Segment Anything Model (SAM) has been proven to be a powerful foundation model for image segmentation tasks. We propose the Task-Aware Low-Rank Adaptation (TA-LoRA) method, which enables SAM to work as a foundation model for multi-task learning.
  arXiv  Detail & Related papers  (2024-03-16T17:02:50Z)
• WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images [8.179859593451285]
  We present WSI-SAM, enhancing Segment Anything Model (SAM) with precise object segmentation capabilities for histopathology images. To fully exploit pretrained knowledge while minimizing training overhead, we keep SAM frozen, introducing only minimal extra parameters. Our model outperforms SAM by 4.1 and 2.5 percent points on a ductal carcinoma in situ (DCIS) segmentation tasks and breast cancer metastasis segmentation task.
  arXiv  Detail & Related papers  (2024-03-14T10:30:43Z)
• TinySAM: Pushing the Envelope for Efficient Segment Anything Model [76.21007576954035]
  We propose a framework to obtain a tiny segment anything model (TinySAM) while maintaining the strong zero-shot performance. We first propose a full-stage knowledge distillation method with hard prompt sampling and hard mask weighting strategy to distill a lightweight student model. We also adapt the post-training quantization to the promptable segmentation task and further reduce the computational cost.
  arXiv  Detail & Related papers  (2023-12-21T12:26:11Z)
• SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model [85.85899655118087]
  We develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS. SAMRS totally possesses 105,090 images and 1,668,241 instances, surpassing existing high-resolution RS segmentation datasets in size by several orders of magnitude.
  arXiv  Detail & Related papers  (2023-05-03T10:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.