Auto-Prompting SAM for Weakly Supervised Landslide Extraction

• URL: http://arxiv.org/abs/2501.13426v1
• Date: Thu, 23 Jan 2025 07:08:48 GMT
• Title: Auto-Prompting SAM for Weakly Supervised Landslide Extraction
• Authors: Jian Wang, Xiaokang Zhang, Xianping Ma, Weikang Yu, Pedram Ghamisi,
• Abstract summary: We propose a simple yet effective method by auto-prompting the Segment Anything Model (SAM) Instead of depending on high-quality class activation maps (CAMs) for pseudo-labeling or fine-tuning SAM, our method directly yields fine-grained segmentation masks from SAM inference through prompt engineering. Experimental results on high-resolution aerial and satellite datasets demonstrate the effectiveness of our method.
• Score: 17.515220489213743
• License:
• Abstract: Weakly supervised landslide extraction aims to identify landslide regions from remote sensing data using models trained with weak labels, particularly image-level labels. However, it is often challenged by the imprecise boundaries of the extracted objects due to the lack of pixel-wise supervision and the properties of landslide objects. To tackle these issues, we propose a simple yet effective method by auto-prompting the Segment Anything Model (SAM), i.e., APSAM. Instead of depending on high-quality class activation maps (CAMs) for pseudo-labeling or fine-tuning SAM, our method directly yields fine-grained segmentation masks from SAM inference through prompt engineering. Specifically, it adaptively generates hybrid prompts from the CAMs obtained by an object localization network. To provide sufficient information for SAM prompting, an adaptive prompt generation (APG) algorithm is designed to fully leverage the visual patterns of CAMs, enabling the efficient generation of pseudo-masks for landslide extraction. These informative prompts are able to identify the extent of landslide areas (box prompts) and denote the centers of landslide objects (point prompts), guiding SAM in landslide segmentation. Experimental results on high-resolution aerial and satellite datasets demonstrate the effectiveness of our method, achieving improvements of at least 3.0\% in F1 score and 3.69\% in IoU compared to other state-of-the-art methods. The source codes and datasets will be available at https://github.com/zxk688.

Related papers

• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes. Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes. We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• MapSAM: Adapting Segment Anything Model for Automated Feature Detection in Historical Maps [6.414068793245697]
  We introduce MapSAM, a parameter-efficient fine-tuning strategy that adapts SAM into a prompt-free and versatile solution for historical map segmentation tasks. Specifically, we employ Weight-Decomposed Low-Rank Adaptation (DoRA) to integrate domain-specific knowledge into the image encoder. We develop an automatic prompt generation process, eliminating the need for manual input.
  arXiv  Detail & Related papers  (2024-11-11T13:18:45Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Images [16.662173255725463]
  We propose a novel Pointly-supervised Segment Anything Model named PointSAM. We conduct experiments on RSI datasets, including WHU, HRSID, and NWPU VHR-10. The results show that our method significantly outperforms direct testing with SAM, SAM2, and other comparison methods.
  arXiv  Detail & Related papers  (2024-09-20T11:02:18Z)
• TrajSSL: Trajectory-Enhanced Semi-Supervised 3D Object Detection [59.498894868956306]
  Pseudo-labeling approaches to semi-supervised learning adopt a teacher-student framework. We leverage pre-trained motion-forecasting models to generate object trajectories on pseudo-labeled data. Our approach improves pseudo-label quality in two distinct manners.
  arXiv  Detail & Related papers  (2024-09-17T05:35:00Z)
• When 3D Partial Points Meets SAM: Tooth Point Cloud Segmentation with Sparse Labels [39.54551717450374]
  Tooth point cloud segmentation is a fundamental task in many orthodontic applications. Recent weakly-supervised alternatives are proposed to use weak labels for 3D segmentation and achieve promising results. We propose a framework named SAMTooth that leverages such capacity to complement the extremely sparse supervision.
  arXiv  Detail & Related papers  (2024-09-03T08:14:56Z)
• Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentation [84.62067728093358]
  Weakly supervised object localization and semantic segmentation aim to localize objects using only image-level labels. New paradigm has emerged by generating a foreground prediction map to achieve pixel-level localization. This paper presents two astonishing experimental observations on the object localization learning process.
  arXiv  Detail & Related papers  (2023-09-22T15:44:10Z)
• Constrained Sampling for Class-Agnostic Weakly Supervised Object Localization [10.542859578763068]
  Self-supervised vision transformers can generate accurate localization maps of the objects in an image. We propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a weakly-supervised object localization model.
  arXiv  Detail & Related papers  (2022-09-09T19:58:38Z)
• Discriminative Sampling of Proposals in Self-Supervised Transformers for Weakly Supervised Object Localization [10.542859578763068]
  Self-supervised vision transformers can generate accurate localization maps of the objects in an image. We propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a weakly-supervised object localization model.
  arXiv  Detail & Related papers  (2022-09-09T18:33:23Z)
• AutoGeoLabel: Automated Label Generation for Geospatial Machine Learning [69.47585818994959]
  We evaluate a big data processing pipeline to auto-generate labels for remote sensing data. We utilize the big geo-data platform IBM PAIRS to dynamically generate such labels in dense urban areas.
  arXiv  Detail & Related papers  (2022-01-31T20:02:22Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.