A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets
Prompt Engineering
- URL: http://arxiv.org/abs/2306.06211v3
- Date: Mon, 3 Jul 2023 08:35:14 GMT
- Title: A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets
Prompt Engineering
- Authors: Chaoning Zhang, Fachrina Dewi Puspitasari, Sheng Zheng, Chenghao Li,
Yu Qiao, Taegoo Kang, Xinru Shan, Chenshuang Zhang, Caiyan Qin, Francois
Rameau, Lik-Hang Lee, Sung-Ho Bae, Choong Seon Hong
- Abstract summary: Segment anything model (SAM) developed by Meta AI Research has attracted significant attention.
With the relevant papers and projects increasing exponentially, it is challenging for the readers to catch up with the development of SAM.
This work conducts the first yet comprehensive survey on SAM.
- Score: 37.68799208121957
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Segment anything model (SAM) developed by Meta AI Research has recently
attracted significant attention. Trained on a large segmentation dataset of
over 1 billion masks, SAM is capable of segmenting any object on a certain
image. In the original SAM work, the authors turned to zero-short transfer
tasks (like edge detection) for evaluating the performance of SAM. Recently,
numerous works have attempted to investigate the performance of SAM in various
scenarios to recognize and segment objects. Moreover, numerous projects have
emerged to show the versatility of SAM as a foundation model by combining it
with other models, like Grounding DINO, Stable Diffusion, ChatGPT, etc. With
the relevant papers and projects increasing exponentially, it is challenging
for the readers to catch up with the development of SAM. To this end, this work
conducts the first yet comprehensive survey on SAM. This is an ongoing project
and we intend to update the manuscript on a regular basis. Therefore, readers
are welcome to contact us if they complete new works related to SAM so that we
can include them in our next version.
Related papers
- AlignSAM: Aligning Segment Anything Model to Open Context via Reinforcement Learning [61.666973416903005]
Segment Anything Model (SAM) has demonstrated its impressive generalization capabilities in open-world scenarios with the guidance of prompts.
We propose a novel framework, termed AlignSAM, designed for automatic prompting for aligning SAM to an open context.
arXiv Detail & Related papers (2024-06-01T16:21:39Z) - MAS-SAM: Segment Any Marine Animal with Aggregated Features [55.91291540810978]
We propose a novel feature learning framework named MAS-SAM for marine animal segmentation.
Our method enables to extract richer marine information from global contextual cues to fine-grained local details.
arXiv Detail & Related papers (2024-04-24T07:38:14Z) - Moving Object Segmentation: All You Need Is SAM (and Flow) [82.78026782967959]
We investigate two models for combining SAM with optical flow that harness the segmentation power of SAM with the ability of flow to discover and group moving objects.
In the first model, we adapt SAM to take optical flow, rather than RGB, as an input. In the second, SAM takes RGB as an input, and flow is used as a segmentation prompt.
These surprisingly simple methods, without any further modifications, outperform all previous approaches by a considerable margin in both single and multi-object benchmarks.
arXiv Detail & Related papers (2024-04-18T17:59:53Z) - Boosting Segment Anything Model Towards Open-Vocabulary Learning [69.42565443181017]
Segment Anything Model (SAM) has emerged as a new paradigmatic vision foundation model.
Despite SAM finding applications and adaptations in various domains, its primary limitation lies in the inability to grasp object semantics.
We present Sambor to seamlessly integrate SAM with the open-vocabulary object detector in an end-to-end framework.
arXiv Detail & Related papers (2023-12-06T17:19:00Z) - On the Robustness of Segment Anything [46.669794757467166]
We aim to study the testing-time robustness of SAM under adversarial scenarios and common corruptions.
We find that SAM exhibits remarkable robustness against various corruptions, except for blur-related corruption.
arXiv Detail & Related papers (2023-05-25T16:28:30Z) - A Comprehensive Survey on Segment Anything Model for Vision and Beyond [7.920790211915402]
It is urgent to design a general class of models, which we term foundation models, trained on broad data.
The recently proposed segment anything model (SAM) has made significant progress in breaking the boundaries of segmentation.
This paper introduces the background and terminology for foundation models including SAM, as well as state-of-the-art methods contemporaneous with SAM.
arXiv Detail & Related papers (2023-05-14T16:23:22Z) - BadSAM: Exploring Security Vulnerabilities of SAM via Backdoor Attacks [16.667225643881782]
We present BadSAM, the first backdoor attack on the image segmentation foundation model.
Our preliminary experiments on the CAMO dataset demonstrate the effectiveness of BadSAM.
arXiv Detail & Related papers (2023-05-05T05:39:12Z) - Segment anything, from space? [8.126645790463266]
"Segment Anything Model" (SAM) can segment objects in input imagery based on cheap input prompts.
SAM usually achieved recognition accuracy similar to, or sometimes exceeding, vision models that had been trained on the target tasks.
We examine whether SAM's performance extends to overhead imagery problems and help guide the community's response to its development.
arXiv Detail & Related papers (2023-04-25T17:14:36Z) - SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in
Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and
More [13.047310918166762]
We propose textbfSAM-Adapter, which incorporates domain-specific information or visual prompts into the segmentation network by using simple yet effective adapters.
We can even outperform task-specific network models and achieve state-of-the-art performance in the task we tested: camouflaged object detection.
arXiv Detail & Related papers (2023-04-18T17:38:54Z) - Segment Anything Is Not Always Perfect: An Investigation of SAM on
Different Real-world Applications [31.31905890353516]
Recently, Meta AI Research approaches a general, promptable Segment Anything Model (SAM) pre-trained on an unprecedentedly large segmentation dataset (SA-1B)
We conduct a series of intriguing investigations into the performance of SAM across various applications, particularly in the fields of natural images, agriculture, manufacturing, remote sensing, and healthcare.
arXiv Detail & Related papers (2023-04-12T10:10:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.