Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects
Cannot Be Easily Detected
- URL: http://arxiv.org/abs/2305.00278v1
- Date: Sat, 29 Apr 2023 15:27:57 GMT
- Title: Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects
Cannot Be Easily Detected
- Authors: Dongsheng Han, Chaoning Zhang, Yu Qiao, Maryam Qamar, Yuna Jung,
SeungKyu Lee, Sung-Ho Bae, Choong Seon Hong
- Abstract summary: As a foundation model in the field of computer vision, SAM (Segment Anything Model) has gained attention for its impressive performance in generic object segmentation.
Despite its strong capability in a wide range of zero-shot transfer tasks, it remains unknown whether SAM can detect things in challenging setups like transparent objects.
- Score: 41.04927631258873
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Meta AI Research has recently released SAM (Segment Anything Model) which is
trained on a large segmentation dataset of over 1 billion masks. As a
foundation model in the field of computer vision, SAM (Segment Anything Model)
has gained attention for its impressive performance in generic object
segmentation. Despite its strong capability in a wide range of zero-shot
transfer tasks, it remains unknown whether SAM can detect things in challenging
setups like transparent objects. In this work, we perform an empirical
evaluation of two glass-related challenging scenarios: mirror and transparent
objects. We found that SAM often fails to detect the glass in both scenarios,
which raises concern for deploying the SAM in safety-critical situations that
have various forms of glass.
Related papers
- Crowd-SAM: SAM as a Smart Annotator for Object Detection in Crowded Scenes [18.244508068200236]
Crowd-SAM is a framework designed to enhance SAM's performance in crowded and occluded scenes.
We introduce an efficient prompt sampler (EPS) and a part-whole discrimination network (PWD-Net) to enhance mask selection and accuracy in crowded scenes.
Crowd-SAM rivals state-of-the-art (SOTA) fully-supervised object detection methods on several benchmarks including CrowdHuman and CityPersons.
arXiv Detail & Related papers (2024-07-16T08:00:01Z) - Moving Object Segmentation: All You Need Is SAM (and Flow) [82.78026782967959]
We investigate two models for combining SAM with optical flow that harness the segmentation power of SAM with the ability of flow to discover and group moving objects.
In the first model, we adapt SAM to take optical flow, rather than RGB, as an input. In the second, SAM takes RGB as an input, and flow is used as a segmentation prompt.
These surprisingly simple methods, without any further modifications, outperform all previous approaches by a considerable margin in both single and multi-object benchmarks.
arXiv Detail & Related papers (2024-04-18T17:59:53Z) - Boosting Segment Anything Model Towards Open-Vocabulary Learning [69.42565443181017]
Segment Anything Model (SAM) has emerged as a new paradigmatic vision foundation model.
Despite SAM finding applications and adaptations in various domains, its primary limitation lies in the inability to grasp object semantics.
We present Sambor to seamlessly integrate SAM with the open-vocabulary object detector in an end-to-end framework.
arXiv Detail & Related papers (2023-12-06T17:19:00Z) - On the Robustness of Segment Anything [46.669794757467166]
We aim to study the testing-time robustness of SAM under adversarial scenarios and common corruptions.
We find that SAM exhibits remarkable robustness against various corruptions, except for blur-related corruption.
arXiv Detail & Related papers (2023-05-25T16:28:30Z) - A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets
Prompt Engineering [37.68799208121957]
Segment anything model (SAM) developed by Meta AI Research has attracted significant attention.
With the relevant papers and projects increasing exponentially, it is challenging for the readers to catch up with the development of SAM.
This work conducts the first yet comprehensive survey on SAM.
arXiv Detail & Related papers (2023-05-12T07:21:59Z) - Attack-SAM: Towards Attacking Segment Anything Model With Adversarial
Examples [68.5719552703438]
Segment Anything Model (SAM) has attracted significant attention recently, due to its impressive performance on various downstream tasks.
Deep vision models are widely recognized as vulnerable to adversarial examples, which fool the model to make wrong predictions with imperceptible perturbation.
This work is the first of its kind to conduct a comprehensive investigation on how to attack SAM with adversarial examples.
arXiv Detail & Related papers (2023-05-01T15:08:17Z) - Can SAM Count Anything? An Empirical Study on SAM Counting [35.42720382193184]
We explore the use of the Segment Anything model (SAM) for the challenging task of few-shot object counting.
We find that SAM's performance is unsatisfactory without further fine-tuning, particularly for small and crowded objects.
arXiv Detail & Related papers (2023-04-21T08:59:48Z) - SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in
Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and
More [13.047310918166762]
We propose textbfSAM-Adapter, which incorporates domain-specific information or visual prompts into the segmentation network by using simple yet effective adapters.
We can even outperform task-specific network models and achieve state-of-the-art performance in the task we tested: camouflaged object detection.
arXiv Detail & Related papers (2023-04-18T17:38:54Z) - SAM Struggles in Concealed Scenes -- Empirical Study on "Segment
Anything" [132.31628334155118]
Segment Anything Model (SAM) fosters the foundation models for computer vision.
In this report, we choose three concealed scenes, i.e., camouflaged animals, industrial defects, and medical lesions, to evaluate SAM under unprompted settings.
Our main observation is that SAM looks unskilled in concealed scenes.
arXiv Detail & Related papers (2023-04-12T17:58:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.