Synthetic Instance Segmentation from Semantic Image Segmentation Masks
- URL: http://arxiv.org/abs/2308.00949v3
- Date: Tue, 31 Oct 2023 04:52:16 GMT
- Title: Synthetic Instance Segmentation from Semantic Image Segmentation Masks
- Authors: Yuchen Shen, Dong Zhang, Yuhui Zheng, Zechao Li, Liyong Fu, Qiaolin Ye
- Abstract summary: We propose a novel paradigm called synthetic instance segmentation (SISeg)
SISeg does not require training a semantic or/and instance segmentation model and avoids the need for instance-level image annotations.
It can achieve competitive results compared to the state-of-the-art fully-supervised instance segmentation methods.
- Score: 37.54211062233899
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, the development of instance segmentation has garnered
significant attention in a wide range of applications. However, the training of
a fully-supervised instance segmentation model requires costly both
instance-level and pixel-level annotations. In contrast, weakly-supervised
instance segmentation methods (i.e., with image-level class labels or point
labels) struggle to satisfy the accuracy and recall requirements of practical
scenarios. In this paper, we propose a novel paradigm called synthetic instance
segmentation (SISeg), which achieves Instance Segmentation results from image
masks predicted using off-the-shelf semantic segmentation models. SISeg does
not require training a semantic or/and instance segmentation model and avoids
the need for instance-level image annotations. Therefore, it is highly
efficient. Specifically, we first obtain a semantic segmentation mask of the
input image via a trained semantic segmentation model. Then, we calculate a
displacement field vector for each pixel based on the segmentation mask, which
can indicate representations belonging to the same class but different
instances, i.e., obtaining the instance-level object information. Finally,
instance segmentation results are obtained after being refined by a learnable
category-agnostic object boundary branch. Extensive experimental results on two
challenging datasets and representative semantic segmentation baselines
(including CNNs and Transformers) demonstrate that SISeg can achieve
competitive results compared to the state-of-the-art fully-supervised instance
segmentation methods without the need for additional human resources or
increased computational costs. The code is available at: SISeg
Related papers
- Unsupervised Universal Image Segmentation [59.0383635597103]
We propose an Unsupervised Universal model (U2Seg) adept at performing various image segmentation tasks.
U2Seg generates pseudo semantic labels for these segmentation tasks via leveraging self-supervised models.
We then self-train the model on these pseudo semantic labels, yielding substantial performance gains.
arXiv Detail & Related papers (2023-12-28T18:59:04Z) - Self-Supervised Instance Segmentation by Grasping [84.2469669256257]
We learn a grasp segmentation model to segment the grasped object from before and after grasp images.
Using the segmented objects, we can "cut" objects from their original scenes and "paste" them into new scenes to generate instance supervision.
We show that our grasp segmentation model provides a 5x error reduction when segmenting grasped objects compared with traditional image subtraction approaches.
arXiv Detail & Related papers (2023-05-10T16:51:36Z) - SIM: Semantic-aware Instance Mask Generation for Box-Supervised Instance
Segmentation [22.930296667684125]
We propose a new box-supervised instance segmentation approach by developing a Semantic-aware Instance Mask (SIM) generation paradigm.
Considering that the semantic-aware prototypes cannot distinguish different instances of the same semantics, we propose a self-correction mechanism.
Extensive experimental results demonstrate the superiority of our proposed SIM approach over other state-of-the-art methods.
arXiv Detail & Related papers (2023-03-14T05:59:25Z) - Scaling up Multi-domain Semantic Segmentation with Sentence Embeddings [81.09026586111811]
We propose an approach to semantic segmentation that achieves state-of-the-art supervised performance when applied in a zero-shot setting.
This is achieved by replacing each class label with a vector-valued embedding of a short paragraph that describes the class.
The resulting merged semantic segmentation dataset of over 2 Million images enables training a model that achieves performance equal to that of state-of-the-art supervised methods on 7 benchmark datasets.
arXiv Detail & Related papers (2022-02-04T07:19:09Z) - Semantic Segmentation In-the-Wild Without Seeing Any Segmentation
Examples [34.97652735163338]
We propose a novel approach for creating semantic segmentation masks for every object.
Our method takes as input the image-level labels of the class categories present in the image.
The output of this stage provides pixel-level pseudo-labels, instead of the manual pixel-level labels required by supervised methods.
arXiv Detail & Related papers (2021-12-06T17:32:38Z) - SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
"instance categories" assigns categories to each pixel within an instance according to the instance's location.
"SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance.
Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
arXiv Detail & Related papers (2021-06-30T09:56:54Z) - Segmenter: Transformer for Semantic Segmentation [79.9887988699159]
We introduce Segmenter, a transformer model for semantic segmentation.
We build on the recent Vision Transformer (ViT) and extend it to semantic segmentation.
It outperforms the state of the art on the challenging ADE20K dataset and performs on-par on Pascal Context and Cityscapes.
arXiv Detail & Related papers (2021-05-12T13:01:44Z) - Learning Panoptic Segmentation from Instance Contours [9.347742071428918]
Panopticpixel aims to provide an understanding of background (stuff) and instances of objects (things) at a pixel level.
It combines the separate tasks of semantic segmentation (level classification) and instance segmentation to build a single unified scene understanding task.
We present a fully convolution neural network that learns instance segmentation from semantic segmentation and instance contours.
arXiv Detail & Related papers (2020-10-16T03:05:48Z) - Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
We propose a novel few-shot semantic segmentation framework based on the prototype representation.
Our key idea is to decompose the holistic class representation into a set of part-aware prototypes.
We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
arXiv Detail & Related papers (2020-07-13T11:03:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.