Self-guided Few-shot Semantic Segmentation for Remote Sensing Imagery Based on Large Vision Models

• URL: http://arxiv.org/abs/2311.13200v1
• Date: Wed, 22 Nov 2023 07:07:55 GMT
• Title: Self-guided Few-shot Semantic Segmentation for Remote Sensing Imagery Based on Large Vision Models
• Authors: Xiyu Qi, Yifan Wu, Yongqiang Mao, Wenhui Zhang, Yidan Zhang
• Abstract summary: This research introduces a structured framework designed for the automation of few-shot semantic segmentation. It utilizes the SAM model and facilitates a more efficient generation of semantically discernible segmentation outcomes. Central to our methodology is a novel automatic prompt learning approach, leveraging prior guided masks to produce coarse pixel-wise prompts for SAM.
• Score: 14.292149307183967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Segment Anything Model (SAM) exhibits remarkable versatility and zero-shot learning abilities, owing largely to its extensive training data (SA-1B). Recognizing SAM's dependency on manual guidance given its category-agnostic nature, we identified unexplored potential within few-shot semantic segmentation tasks for remote sensing imagery. This research introduces a structured framework designed for the automation of few-shot semantic segmentation. It utilizes the SAM model and facilitates a more efficient generation of semantically discernible segmentation outcomes. Central to our methodology is a novel automatic prompt learning approach, leveraging prior guided masks to produce coarse pixel-wise prompts for SAM. Extensive experiments on the DLRSD datasets underline the superiority of our approach, outperforming other available few-shot methodologies.

Related papers

• Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images [11.845626002236772]
  We introduce a foundation model derived from DirectSAM, termed DirectSAM-RS, which inherits the strong segmentation capability acquired from natural images. This dataset comprises over 34k image-text-contour triplets, making it at least 30 times larger than individual dataset. We evaluate the DirectSAM-RS in both zero-shot and fine-tuning setting, and demonstrate that it achieves state-of-the-art performance across several downstream benchmarks.
  arXiv  Detail & Related papers  (2024-10-08T16:55:42Z)
• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• Tuning a SAM-Based Model with Multi-Cognitive Visual Adapter to Remote Sensing Instance Segmentation [4.6570959687411975]
  The Segment Anything Model (SAM) demonstrates exceptional generalization capabilities. SAM's lack of pretraining on massive remote sensing images and its interactive structure limit its automatic mask prediction capabilities. A Multi- cognitive SAM-Based Instance Model (MC-SAM SEG) is introduced to employ SAM on remote sensing domain. The proposed method named MC-SAM SEG extracts high-quality features by fine-tuning the SAM-Mona encoder along with a feature aggregator.
  arXiv  Detail & Related papers  (2024-08-16T07:23:22Z)
• AlignSAM: Aligning Segment Anything Model to Open Context via Reinforcement Learning [61.666973416903005]
  Segment Anything Model (SAM) has demonstrated its impressive generalization capabilities in open-world scenarios with the guidance of prompts. We propose a novel framework, termed AlignSAM, designed for automatic prompting for aligning SAM to an open context.
  arXiv  Detail & Related papers  (2024-06-01T16:21:39Z)
• Boosting Segment Anything Model Towards Open-Vocabulary Learning [69.42565443181017]
  Segment Anything Model (SAM) has emerged as a new paradigmatic vision foundation model. Despite SAM finding applications and adaptations in various domains, its primary limitation lies in the inability to grasp object semantics. We present Sambor to seamlessly integrate SAM with the open-vocabulary object detector in an end-to-end framework.
  arXiv  Detail & Related papers  (2023-12-06T17:19:00Z)
• Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM) [8.529233820032678]
  The Segment Anything Model (SAM) is the first foundation model for image segmentation. In this study, we evaluate SAM's ability to segment features from eye images recorded in virtual reality setups. Our investigation centers on SAM's zero-shot learning abilities and the effectiveness of prompts like bounding boxes or point clicks.
  arXiv  Detail & Related papers  (2023-11-14T11:05:08Z)
• Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
  We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
  arXiv  Detail & Related papers  (2023-07-10T17:59:40Z)
• RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation [53.4319652364256]
  This paper presents the RefSAM model, which explores the potential of SAM for referring video object segmentation. Our proposed approach adapts the original SAM model to enhance cross-modality learning by employing a lightweight Cross-RValModal. We employ a parameter-efficient tuning strategy to align and fuse the language and vision features effectively.
  arXiv  Detail & Related papers  (2023-07-03T13:21:58Z)
• The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot [6.500451285898152]
  This study aims to advance the application of the Segment Anything Model (SAM) in remote sensing image analysis. SAM is known for its exceptional generalization capabilities and zero-shot learning. Despite the limitations encountered with lower spatial resolution images, SAM exhibits promising adaptability to remote sensing data analysis.
  arXiv  Detail & Related papers  (2023-06-29T01:49:33Z)
• RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model [29.42043345787285]
  We propose a method to learn the generation of appropriate prompts for Segment Anything Model (SAM) This enables SAM to produce semantically discernible segmentation results for remote sensing images. We also propose several ongoing derivatives for instance segmentation tasks, drawing on recent advancements within the SAM community, and compare their performance with RSPrompter.
  arXiv  Detail & Related papers  (2023-06-28T14:51:34Z)
• Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models [92.92095626286223]
  We develop an automatic procedure for finding directions that lead to foreground-background image separation. We use these directions to train an image segmentation model without human supervision.
  arXiv  Detail & Related papers  (2021-05-17T19:34:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.