Towards Unsupervised Open World Semantic Segmentation

• URL: http://arxiv.org/abs/2201.01073v1
• Date: Tue, 4 Jan 2022 10:29:34 GMT
• Title: Towards Unsupervised Open World Semantic Segmentation
• Authors: Svenja Uhlemeyer, Matthias Rottmann, Hanno Gottschalk
• Abstract summary: We introduce a method where unknown objects are clustered based on visual similarity. connected components of a predicted semantic segmentation are assessed by a segmentation quality estimate. We demonstrate that, without access to ground truth and even with few data, a DNN's class space can be extended by a novel class.
• Score: 6.445605125467575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For the semantic segmentation of images, state-of-the-art deep neural networks (DNNs) achieve high segmentation accuracy if that task is restricted to a closed set of classes. However, as of now DNNs have limited ability to operate in an open world, where they are tasked to identify pixels belonging to unknown objects and eventually to learn novel classes, incrementally. Humans have the capability to say: I don't know what that is, but I've already seen something like that. Therefore, it is desirable to perform such an incremental learning task in an unsupervised fashion. We introduce a method where unknown objects are clustered based on visual similarity. Those clusters are utilized to define new classes and serve as training data for unsupervised incremental learning. More precisely, the connected components of a predicted semantic segmentation are assessed by a segmentation quality estimate. connected components with a low estimated prediction quality are candidates for a subsequent clustering. Additionally, the component-wise quality assessment allows for obtaining predicted segmentation masks for the image regions potentially containing unknown objects. The respective pixels of such masks are pseudo-labeled and afterwards used for re-training the DNN, i.e., without the use of ground truth generated by humans. In our experiments we demonstrate that, without access to ground truth and even with few data, a DNN's class space can be extended by a novel class, achieving considerable segmentation accuracy.

Related papers

• NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation [87.50830107535533]
  Semi-supervised semantic segmentation involves assigning pixel-wise labels to unlabeled images at training time. Current approaches to semi-supervised semantic segmentation work by predicting pseudo-labels for each pixel from a class-wise probability distribution output by a model. In this work, we move one step forward by adapting NPs to semi-supervised semantic segmentation, resulting in a new model called NP-SemiSeg.
  arXiv  Detail & Related papers  (2023-08-05T12:42:15Z)
• Evidential Deep Learning for Class-Incremental Semantic Segmentation [15.563703446465823]
  Class-Incremental Learning is a challenging problem in machine learning that aims to extend previously trained neural networks with new classes. In this paper, we address the problem of how to model unlabeled classes while avoiding spurious feature clustering of future uncorrelated classes. Our method factorizes the problem into a separate foreground class probability, calculated by the expected value of the Dirichlet distribution, and an unknown class (background) probability corresponding to the uncertainty of the estimate.
  arXiv  Detail & Related papers  (2022-12-06T10:13:30Z)
• Visual Recognition with Deep Nearest Centroids [57.35144702563746]
  We devise deep nearest centroids (DNC), a conceptually elegant yet surprisingly effective network for large-scale visual recognition. Compared with parametric counterparts, DNC performs better on image classification (CIFAR-10, ImageNet) and greatly boots pixel recognition (ADE20K, Cityscapes)
  arXiv  Detail & Related papers  (2022-09-15T15:47:31Z)
• Segmenting Known Objects and Unseen Unknowns without Prior Knowledge [86.46204148650328]
  holistic segmentation aims to identify and separate objects of unseen, unknown categories into instances without any prior knowledge about them. We tackle this new problem with U3HS, which finds unknowns as highly uncertain regions and clusters their corresponding instance-aware embeddings into individual objects. Experiments on public data from MS, Cityscapes, and Lost&Found demonstrate the effectiveness of U3HS.
  arXiv  Detail & Related papers  (2022-09-12T16:59:36Z)
• CvS: Classification via Segmentation For Small Datasets [52.821178654631254]
  This paper presents CvS, a cost-effective classifier for small datasets that derives the classification labels from predicting the segmentation maps. We evaluate the effectiveness of our framework on diverse problems showing that CvS is able to achieve much higher classification results compared to previous methods when given only a handful of examples.
  arXiv  Detail & Related papers  (2021-10-29T18:41:15Z)
• Selective Pseudo-label Clustering [42.19193184852487]
  Deep neural networks (DNNs) offer a means of addressing the challenging task of clustering high-dimensional data. We propose selective pseudo-label clustering, which uses only the most confident pseudo-labels for training theDNN. New approach achieves a state-of-the-art performance on three popular image datasets.
  arXiv  Detail & Related papers  (2021-07-22T13:56:53Z)
• LRGNet: Learnable Region Growing for Class-Agnostic Point Cloud Segmentation [19.915593390338337]
  This research proposes a learnable region growing method for class-agnostic point cloud segmentation. The proposed method is able to segment any class of objects using a single deep neural network without any assumptions about their shapes and sizes.
  arXiv  Detail & Related papers  (2021-03-16T15:58:01Z)
• Fool Me Once: Robust Selective Segmentation via Out-of-Distribution Detection with Contrastive Learning [27.705683228657175]
  We train a network to simultaneously perform segmentation and pixel-wise Out-of-Distribution (OoD) detection. This is made possible by leveraging an OoD dataset with a novel contrastive objective and data augmentation scheme. We show that by selectively segmenting scenes based on what is predicted as OoD, we can increase the segmentation accuracy by an IoU of 0.2 with respect to alternative techniques.
  arXiv  Detail & Related papers  (2021-03-01T09:38:40Z)
• Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals [78.12377360145078]
  We introduce a novel two-step framework that adopts a predetermined prior in a contrastive optimization objective to learn pixel embeddings. This marks a large deviation from existing works that relied on proxy tasks or end-to-end clustering. In particular, when fine-tuning the learned representations using just 1% of labeled examples on PASCAL, we outperform supervised ImageNet pre-training by 7.1% mIoU.
  arXiv  Detail & Related papers  (2021-02-11T18:54:47Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Region Growing with Convolutional Neural Networks for Biomedical Image Segmentation [1.5469452301122177]
  We present a methodology that uses convolutional neural networks (CNNs) for segmentation by iteratively growing predicted mask regions in each coordinate direction. We use a threshold on the CNN probability scores to determine whether pixels are added to the region and the iteration continues until no new pixels are added to the region. Our method is able to achieve high segmentation accuracy and preserve biologically realistic morphological features while leveraging small amounts of training data and maintaining computational efficiency.
  arXiv  Detail & Related papers  (2020-09-23T17:53:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.