Segment Anything

• URL: http://arxiv.org/abs/2304.02643v1
• Date: Wed, 5 Apr 2023 17:59:46 GMT
• Title: Segment Anything
• Authors: Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Doll\'ar, Ross Girshick
• Abstract summary: We build the largest segmentation dataset to date, with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive.
• Score: 108.16489338211093
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date (by far), with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive -- often competitive with or even superior to prior fully supervised results. We are releasing the Segment Anything Model (SAM) and corresponding dataset (SA-1B) of 1B masks and 11M images at https://segment-anything.com to foster research into foundation models for computer vision.

Related papers

• UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation [64.01742988773745]
  An increasing privacy concern exists regarding training large-scale image segmentation models on unauthorized private data. We exploit the concept of unlearnable examples to make images unusable to model training by generating and adding unlearnable noise into the original images. We empirically verify the effectiveness of UnSeg across 6 mainstream image segmentation tasks, 10 widely used datasets, and 7 different network architectures.
  arXiv  Detail & Related papers  (2024-10-13T16:34:46Z)
• SAM 2: Segment Anything in Images and Videos [63.44869623822368]
  We present Segment Anything Model 2 (SAM 2), a foundation model towards solving promptable visual segmentation in images and videos. We build a data engine, which improves model and data via user interaction, to collect the largest video segmentation dataset to date. Our model is a simple transformer architecture with streaming memory for real-time video processing.
  arXiv  Detail & Related papers  (2024-08-01T17:00:08Z)
• Toffee: Efficient Million-Scale Dataset Construction for Subject-Driven Text-to-Image Generation [58.09421301921607]
  We construct the first large-scale dataset for subject-driven image editing and generation. Our dataset is 5 times the size of previous largest dataset, yet our cost is tens of thousands of GPU hours lower.
  arXiv  Detail & Related papers  (2024-06-13T16:40:39Z)
• EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything [36.553867358541154]
  Segment Anything Model (SAM) has emerged as a powerful tool for numerous vision applications. We propose EfficientSAMs, light-weight SAM models that exhibits decent performance with largely reduced complexity. Our idea is based on leveraging masked image pretraining, SAMI, which learns to reconstruct features from SAM image encoder for effective visual representation learning.
  arXiv  Detail & Related papers  (2023-12-01T18:31:00Z)
• Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
  We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
  arXiv  Detail & Related papers  (2023-07-10T17:59:40Z)
• Segment Anything in High Quality [116.39405160133315]
  We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We show the efficacy of HQ-SAM in a suite of 10 diverse segmentation datasets across different downstream tasks, where 8 out of them are evaluated in a zero-shot transfer protocol.
  arXiv  Detail & Related papers  (2023-06-02T14:23:59Z)
• SAM on Medical Images: A Comprehensive Study on Three Prompt Modes [12.42280534113305]
  The Segment Anything Model (SAM) made an eye-catching debut recently and inspired many researchers to explore its potential and limitation in terms of zero-shot generalization capability. In this paper, we evaluate whether SAM has the potential to become the foundation model for medical image segmentation tasks. We also explore what kind of prompt can lead to the best zero-shot performance with different modalities.
  arXiv  Detail & Related papers  (2023-04-28T18:18:07Z)
• Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Model [36.015065439244495]
  The Segment Anything Model (SAM) is a recently developed large model for general-purpose segmentation for computer vision tasks. SAM was trained using 11 million images with over 1 billion masks and can produce segmentation results for a wide range of objects in natural scene images. This paper shows that although SAM does not immediately give high-quality segmentation for medical image data, its generated masks, features, and stability scores are useful for building and training better medical image segmentation models.
  arXiv  Detail & Related papers  (2023-04-22T07:11:53Z)
• Learning to "Segment Anything" in Thermal Infrared Images through Knowledge Distillation with a Large Scale Dataset SATIR [15.198798677908615]
  The Segment Anything Model (SAM) is a promptable segmentation model recently introduced by Meta AI. We propose a framework that utilizes SAM to generate pseudo labels for pretraining thermal infrared image segmentation tasks. Our framework presents a novel approach to collaborate with models trained with large data like SAM to address problems in special fields.
  arXiv  Detail & Related papers  (2023-04-17T03:27:10Z)
• Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging [12.533476185972527]
  The segment anything model (SAM) was released as a foundation model for image segmentation. We evaluate the zero-shot segmentation performance of SAM model on representative segmentation tasks on whole slide imaging (WSI) The results suggest that the zero-shot SAM model achieves remarkable segmentation performance for large connected objects.
  arXiv  Detail & Related papers  (2023-04-09T04:06:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.