Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations

• URL: http://arxiv.org/abs/2207.05027v2
• Date: Sun, 30 Apr 2023 17:39:42 GMT
• Title: Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations
• Authors: Andrii Zadaianchuk, Matthaeus Kleindessner, Yi Zhu, Francesco Locatello, Thomas Brox
• Abstract summary: We show that recent advances in self-supervised feature learning enable unsupervised object discovery and semantic segmentation with a performance that matches the state of the field on supervised semantic segmentation 10 years ago. We propose a methodology based on unsupervised saliency masks and self-supervised feature clustering to kickstart object discovery followed by training a semantic segmentation network on pseudo-labels to bootstrap the system on images with multiple objects.
• Score: 54.32381783582341
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we show that recent advances in self-supervised feature learning enable unsupervised object discovery and semantic segmentation with a performance that matches the state of the field on supervised semantic segmentation 10 years ago. We propose a methodology based on unsupervised saliency masks and self-supervised feature clustering to kickstart object discovery followed by training a semantic segmentation network on pseudo-labels to bootstrap the system on images with multiple objects. We present results on PASCAL VOC that go far beyond the current state of the art (50.0 mIoU), and we report for the first time results on MS COCO for the whole set of 81 classes: our method discovers 34 categories with more than $20\%$ IoU, while obtaining an average IoU of 19.6 for all 81 categories.

Related papers

• SOHES: Self-supervised Open-world Hierarchical Entity Segmentation [82.45303116125021]
  This work presents Self-supervised Open-world Hierarchical Entities (SOHES), a novel approach that eliminates the need for human annotations. We produce abundant high-quality pseudo-labels through visual feature clustering, and rectify the noises in pseudo-labels via a teacher- mutual-learning procedure. Using raw images as the sole training data, our method achieves unprecedented performance in self-supervised open-world segmentation.
  arXiv  Detail & Related papers  (2024-04-18T17:59:46Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Semi-supervised learning made simple with self-supervised clustering [65.98152950607707]
  Self-supervised learning models have been shown to learn rich visual representations without requiring human annotations. We propose a conceptually simple yet empirically powerful approach to turn clustering-based self-supervised methods into semi-supervised learners.
  arXiv  Detail & Related papers  (2023-06-13T01:09:18Z)
• MOVE: Unsupervised Movable Object Segmentation and Detection [32.73565093619594]
  MOVE is a method to segment objects without any form of supervision. It exploits the fact that foreground objects can be shifted locally relative to their initial position. It gives an average CorLoc improvement of 7.2% over the SotA.
  arXiv  Detail & Related papers  (2022-10-14T16:05:46Z)
• FreeSOLO: Learning to Segment Objects without Annotations [191.82134817449528]
  We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO. Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-02-24T16:31:44Z)
• Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision [3.841232411073827]
  We propose a method that can perform unsupervised discovery of long-tail categories in instance segmentation. Our model is able to discover novel and more fine-grained objects than the common categories. We show that the model achieves competitive quantitative results on LVIS as compared to the supervised and partially supervised methods.
  arXiv  Detail & Related papers  (2021-04-02T22:05:03Z)
• Weakly-Supervised Semantic Segmentation via Sub-category Exploration [73.03956876752868]
  We propose a simple yet effective approach to enforce the network to pay attention to other parts of an object. Specifically, we perform clustering on image features to generate pseudo sub-categories labels within each annotated parent class. We conduct extensive analysis to validate the proposed method and show that our approach performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-08-03T20:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.