Zero-Shot Edge Detection with SCESAME: Spectral Clustering-based Ensemble for Segment Anything Model Estimation

• URL: http://arxiv.org/abs/2308.13779v2
• Date: Sat, 18 Nov 2023 12:41:22 GMT
• Title: Zero-Shot Edge Detection with SCESAME: Spectral Clustering-based Ensemble for Segment Anything Model Estimation
• Authors: Hiroaki Yamagiwa, Yusuke Takase, Hiroyuki Kambe, Ryosuke Nakamoto
• Abstract summary: This paper proposes a novel zero-shot edge detection with SCESAME, based on the recently proposed Segment Anything Model (SAM) AMG can be applied to edge detection, but suffers from the problem of overdetecting edges. We performed edge detection experiments on two datasets, BSDS500 and NYUDv2.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a novel zero-shot edge detection with SCESAME, which stands for Spectral Clustering-based Ensemble for Segment Anything Model Estimation, based on the recently proposed Segment Anything Model (SAM). SAM is a foundation model for segmentation tasks, and one of the interesting applications of SAM is Automatic Mask Generation (AMG), which generates zero-shot segmentation masks of an entire image. AMG can be applied to edge detection, but suffers from the problem of overdetecting edges. Edge detection with SCESAME overcomes this problem by three steps: (1) eliminating small generated masks, (2) combining masks by spectral clustering, taking into account mask positions and overlaps, and (3) removing artifacts after edge detection. We performed edge detection experiments on two datasets, BSDS500 and NYUDv2. Although our zero-shot approach is simple, the experimental results on BSDS500 showed almost identical performance to human performance and CNN-based methods from seven years ago. In the NYUDv2 experiments, it performed almost as well as recent CNN-based methods. These results indicate that our method effectively enhances the utility of SAM and can be a new direction in zero-shot edge detection methods.

Related papers

• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Images [16.662173255725463]
  We propose a novel Pointly-supervised Segment Anything Model named PointSAM. We conduct experiments on RSI datasets, including WHU, HRSID, and NWPU VHR-10. The results show that our method significantly outperforms direct testing with SAM, SAM2, and other comparison methods.
  arXiv  Detail & Related papers  (2024-09-20T11:02:18Z)
• From Generalization to Precision: Exploring SAM for Tool Segmentation in Surgical Environments [7.01085327371458]
  We argue that Segment Anything Model drastically over-segment images with high corruption levels, resulting in degraded performance. We employ the ground-truth tool mask to analyze the results of SAM when the best single mask is selected as prediction. We analyze the Endovis18 and Endovis17 instrument segmentation datasets using synthetic corruptions of various strengths and an In-House dataset featuring counterfactually created real-world corruptions.
  arXiv  Detail & Related papers  (2024-02-28T01:33:49Z)
• Monte Carlo Linear Clustering with Single-Point Supervision is Enough for Infrared Small Target Detection [48.707233614642796]
  Single-frame infrared small target (SIRST) detection aims at separating small targets from clutter backgrounds on infrared images. Deep learning based methods have achieved promising performance on SIRST detection, but at the cost of a large amount of training data. We propose the first method to achieve SIRST detection with single-point supervision.
  arXiv  Detail & Related papers  (2023-04-10T08:04:05Z)
• Cut and Learn for Unsupervised Object Detection and Instance Segmentation [65.43627672225624]
  Cut-and-LEaRn (CutLER) is a simple approach for training unsupervised object detection and segmentation models. CutLER is a zero-shot unsupervised detector and improves detection performance AP50 by over 2.7 times on 11 benchmarks.
  arXiv  Detail & Related papers  (2023-01-26T18:57:13Z)
• DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows [52.31831255787147]
  We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA) Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution. Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
  arXiv  Detail & Related papers  (2021-05-30T22:07:13Z)
• Object Detection Made Simpler by Eliminating Heuristic NMS [70.93004137521946]
  We show a simple NMS-free, end-to-end object detection framework. We attain on par or even improved detection accuracy compared with the original one-stage detector.
  arXiv  Detail & Related papers  (2021-01-28T02:38:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.