Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects Cannot Be Easily Detected

• URL: http://arxiv.org/abs/2305.00278v1
• Date: Sat, 29 Apr 2023 15:27:57 GMT
• Title: Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects Cannot Be Easily Detected
• Authors: Dongsheng Han, Chaoning Zhang, Yu Qiao, Maryam Qamar, Yuna Jung, SeungKyu Lee, Sung-Ho Bae, Choong Seon Hong
• Abstract summary: As a foundation model in the field of computer vision, SAM (Segment Anything Model) has gained attention for its impressive performance in generic object segmentation. Despite its strong capability in a wide range of zero-shot transfer tasks, it remains unknown whether SAM can detect things in challenging setups like transparent objects.
• Score: 41.04927631258873
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Meta AI Research has recently released SAM (Segment Anything Model) which is trained on a large segmentation dataset of over 1 billion masks. As a foundation model in the field of computer vision, SAM (Segment Anything Model) has gained attention for its impressive performance in generic object segmentation. Despite its strong capability in a wide range of zero-shot transfer tasks, it remains unknown whether SAM can detect things in challenging setups like transparent objects. In this work, we perform an empirical evaluation of two glass-related challenging scenarios: mirror and transparent objects. We found that SAM often fails to detect the glass in both scenarios, which raises concern for deploying the SAM in safety-critical situations that have various forms of glass.

Related papers

• Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection [58.241593208031816]
  Segment Anything Model (SAM) has been proposed as a visual fundamental model, which gives strong segmentation and generalization capabilities. We propose a Multi-scale and Detail-enhanced SAM (MDSAM) for Salient Object Detection (SOD) Experimental results demonstrate the superior performance of our model on multiple SOD datasets.
  arXiv  Detail & Related papers  (2024-08-08T09:09:37Z)
• Evaluating SAM2's Role in Camouflaged Object Detection: From SAM to SAM2 [10.751277821864916]
  Report reveals a decline in SAM2's ability to perceive different objects in images without prompts in its auto mode. Specifically, we employ the challenging task of camouflaged object detection to assess this performance decrease.
  arXiv  Detail & Related papers  (2024-07-31T13:32:10Z)
• Crowd-SAM: SAM as a Smart Annotator for Object Detection in Crowded Scenes [18.244508068200236]
  Crowd-SAM is a framework designed to enhance SAM's performance in crowded and occluded scenes. We introduce an efficient prompt sampler (EPS) and a part-whole discrimination network (PWD-Net) to enhance mask selection and accuracy in crowded scenes. Crowd-SAM rivals state-of-the-art (SOTA) fully-supervised object detection methods on several benchmarks including CrowdHuman and CityPersons.
  arXiv  Detail & Related papers  (2024-07-16T08:00:01Z)
• Moving Object Segmentation: All You Need Is SAM (and Flow) [82.78026782967959]
  We investigate two models for combining SAM with optical flow that harness the segmentation power of SAM with the ability of flow to discover and group moving objects. In the first model, we adapt SAM to take optical flow, rather than RGB, as an input. In the second, SAM takes RGB as an input, and flow is used as a segmentation prompt. These surprisingly simple methods, without any further modifications, outperform all previous approaches by a considerable margin in both single and multi-object benchmarks.
  arXiv  Detail & Related papers  (2024-04-18T17:59:53Z)
• On the Robustness of Segment Anything [46.669794757467166]
  We aim to study the testing-time robustness of SAM under adversarial scenarios and common corruptions. We find that SAM exhibits remarkable robustness against various corruptions, except for blur-related corruption.
  arXiv  Detail & Related papers  (2023-05-25T16:28:30Z)
• Attack-SAM: Towards Attacking Segment Anything Model With Adversarial Examples [68.5719552703438]
  Segment Anything Model (SAM) has attracted significant attention recently, due to its impressive performance on various downstream tasks. Deep vision models are widely recognized as vulnerable to adversarial examples, which fool the model to make wrong predictions with imperceptible perturbation. This work is the first of its kind to conduct a comprehensive investigation on how to attack SAM with adversarial examples.
  arXiv  Detail & Related papers  (2023-05-01T15:08:17Z)
• Can SAM Count Anything? An Empirical Study on SAM Counting [35.42720382193184]
  We explore the use of the Segment Anything model (SAM) for the challenging task of few-shot object counting. We find that SAM's performance is unsatisfactory without further fine-tuning, particularly for small and crowded objects.
  arXiv  Detail & Related papers  (2023-04-21T08:59:48Z)
• SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More [13.047310918166762]
  We propose textbfSAM-Adapter, which incorporates domain-specific information or visual prompts into the segmentation network by using simple yet effective adapters. We can even outperform task-specific network models and achieve state-of-the-art performance in the task we tested: camouflaged object detection.
  arXiv  Detail & Related papers  (2023-04-18T17:38:54Z)
• SAM Struggles in Concealed Scenes -- Empirical Study on "Segment Anything" [132.31628334155118]
  Segment Anything Model (SAM) fosters the foundation models for computer vision. In this report, we choose three concealed scenes, i.e., camouflaged animals, industrial defects, and medical lesions, to evaluate SAM under unprompted settings. Our main observation is that SAM looks unskilled in concealed scenes.
  arXiv  Detail & Related papers  (2023-04-12T17:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.