Exemplar-Based Open-Set Panoptic Segmentation Network

• URL: http://arxiv.org/abs/2105.08336v2
• Date: Wed, 19 May 2021 00:38:26 GMT
• Title: Exemplar-Based Open-Set Panoptic Segmentation Network
• Authors: Jaedong Hwang, Seoung Wug Oh, Joon-Young Lee, Bohyung Han
• Abstract summary: We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task. We investigate the practical challenges of the task and construct a benchmark on top of an existing dataset, COCO. We propose a novel exemplar-based open-set panoptic segmentation network (EOPSN) inspired by exemplar theory.
• Score: 79.99748041746592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task. This task requires performing panoptic segmentation for not only known classes but also unknown ones that have not been acknowledged during training. We investigate the practical challenges of the task and construct a benchmark on top of an existing dataset, COCO. In addition, we propose a novel exemplar-based open-set panoptic segmentation network (EOPSN) inspired by exemplar theory. Our approach identifies a new class based on exemplars, which are identified by clustering and employed as pseudo-ground-truths. The size of each class increases by mining new exemplars based on the similarities to the existing ones associated with the class. We evaluate EOPSN on the proposed benchmark and demonstrate the effectiveness of our proposals. The primary goal of our work is to draw the attention of the community to the recognition in the open-world scenarios. The implementation of our algorithm is available on the project webpage: https://cv.snu.ac.kr/research/EOPSN.

Related papers

• Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervision [83.57156368908836]
  We propose a novel approach for open world instance segmentation called bottom-Up and top-Down Open-world (UDOS) UDOS first predicts parts of objects using a top-down network trained with weak supervision from bottom-up segmentations. UDOS enjoys both the speed and efficiency from the topdown architectures and the ability to unseen categories from bottom-up supervision.
  arXiv  Detail & Related papers  (2023-03-09T18:55:03Z)
• ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Data [13.978966783993146]
  Open-world Instance (OIS) is a challenging task that aims to accurately segment every object instance appearing in the current observation. This is important for safety-critical applications such as robust autonomous navigation. We present a flexible and effective OIS framework for LiDAR point cloud that can accurately segment both known and unknown instances.
  arXiv  Detail & Related papers  (2023-03-08T03:22:11Z)
• Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity [59.1823948436411]
  We propose a novel approach for mask proposals, Generic Grouping Networks (GGNs) Our approach combines a local measure of pixel affinity with instance-level mask supervision, producing a training regimen designed to make the model as generic as the data diversity allows.
  arXiv  Detail & Related papers  (2022-04-12T22:37:49Z)
• GOSS: Towards Generalized Open-set Semantic Segmentation [38.00665104750745]
  We present and study a new image segmentation task, called Generalized Open-set Semantic (GOSS) GOSS classifies pixels as belonging to known classes, and clusters (or groups) of pixels of unknown class are labelled as such. We believe our new GOSS task can produce an expressive image understanding for future research.
  arXiv  Detail & Related papers  (2022-03-23T01:03:06Z)
• Novel Class Discovery in Semantic Segmentation [104.30729847367104]
  We introduce a new setting of Novel Class Discovery in Semantic (NCDSS) It aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image. We propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels.
  arXiv  Detail & Related papers  (2021-12-03T13:31:59Z)
• A Technical Survey and Evaluation of Traditional Point Cloud Clustering Methods for LiDAR Panoptic Segmentation [11.138159123596669]
  LiDAR panoptic segmentation is a newly proposed technical task for autonomous driving. We propose a hybrid method with an existing semantic segmentation network to extract semantic information. We show a state-of-the-art performance among all published end-to-end deep learning solutions on the panoptic segmentation leaderboard.
  arXiv  Detail & Related papers  (2021-08-21T14:59:02Z)
• GP-S3Net: Graph-based Panoptic Sparse Semantic Segmentation Network [1.9949920338542213]
  GP-S3Net is a proposal-free approach in which no object proposals are needed to identify the objects. Our new design consists of a novel instance-level network to process the semantic results. Extensive experiments demonstrate that GP-S3Net outperforms the current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-08-18T21:49:58Z)
• Region Comparison Network for Interpretable Few-shot Image Classification [97.97902360117368]
  Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes. We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works. We also present a new way to generalize the interpretability from the level of tasks to categories.
  arXiv  Detail & Related papers  (2020-09-08T07:29:05Z)
• Graph Prototypical Networks for Few-shot Learning on Attributed Networks [72.31180045017835]
  We propose a graph meta-learning framework -- Graph Prototypical Networks (GPN) GPN is able to perform textitmeta-learning on an attributed network and derive a highly generalizable model for handling the target classification task.
  arXiv  Detail & Related papers  (2020-06-23T04:13:23Z)
• Towards Bounding-Box Free Panoptic Segmentation [16.4548904544277]
  We introduce a new Bounding-Box Free Network (BBFNet) for panoptic segmentation. BBFNet predicts coarse watershed levels and uses them to detect large instance candidates where boundaries are well defined. For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects.
  arXiv  Detail & Related papers  (2020-02-18T16:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.