PUPS: Point Cloud Unified Panoptic Segmentation

• URL: http://arxiv.org/abs/2302.06185v1
• Date: Mon, 13 Feb 2023 08:42:41 GMT
• Title: PUPS: Point Cloud Unified Panoptic Segmentation
• Authors: Shihao Su, Jianyun Xu, Huanyu Wang, Zhenwei Miao, Xin Zhan, Dayang Hao, Xi Li
• Abstract summary: We propose a simple but effective point cloud unified panoptic segmentation (PUPS) framework. PUPS uses a set of point-level classifiers to directly predict semantic and instance groupings in an end-to-end manner. PUPS achieves 1st place on the leader board of Semantic KITTI panoptic segmentation task and state-of-the-art results on nuScenes.
• Score: 13.668363631123649
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Point cloud panoptic segmentation is a challenging task that seeks a holistic solution for both semantic and instance segmentation to predict groupings of coherent points. Previous approaches treat semantic and instance segmentation as surrogate tasks, and they either use clustering methods or bounding boxes to gather instance groupings with costly computation and hand-crafted designs in the instance segmentation task. In this paper, we propose a simple but effective point cloud unified panoptic segmentation (PUPS) framework, which use a set of point-level classifiers to directly predict semantic and instance groupings in an end-to-end manner. To realize PUPS, we introduce bipartite matching to our training pipeline so that our classifiers are able to exclusively predict groupings of instances, getting rid of hand-crafted designs, e.g. anchors and Non-Maximum Suppression (NMS). In order to achieve better grouping results, we utilize a transformer decoder to iteratively refine the point classifiers and develop a context-aware CutMix augmentation to overcome the class imbalance problem. As a result, PUPS achieves 1st place on the leader board of SemanticKITTI panoptic segmentation task and state-of-the-art results on nuScenes.

Related papers

• Uncovering Prototypical Knowledge for Weakly Open-Vocabulary Semantic Segmentation [59.37587762543934]
  This paper studies the problem of weakly open-vocabulary semantic segmentation (WOVSS) Existing methods suffer from a granularity inconsistency regarding the usage of group tokens. We propose the prototypical guidance network (PGSeg) that incorporates multi-modal regularization.
  arXiv  Detail & Related papers  (2023-10-29T13:18:00Z)
• PANet: LiDAR Panoptic Segmentation with Sparse Instance Proposal and Aggregation [15.664835767712775]
  This work proposes a new LPS framework named PANet to eliminate the dependency on the offset branch. PaNet achieves state-of-the-art performance among published works on the Semantic KITII validation and nuScenes validation for the panoptic segmentation task.
  arXiv  Detail & Related papers  (2023-06-27T10:02:28Z)
• CLUSTSEG: Clustering for Universal Segmentation [56.58677563046506]
  CLUSTSEG is a general, transformer-based framework for image segmentation. It tackles different image segmentation tasks (i.e., superpixel, semantic, instance, and panoptic) through a unified neural clustering scheme.
  arXiv  Detail & Related papers  (2023-05-03T15:31:16Z)
• 1st Place Solution for PSG competition with ECCV'22 SenseHuman Workshop [1.5362025549031049]
  Panoptic Scene Graph (PSG) generation aims to generate scene graph representations based on panoptic segmentation instead of rigid bounding boxes. We propose GRNet, a Global Relation Network in two-stage paradigm, where the pre-extracted local object features and their corresponding masks are fed into a transformer with class embeddings. We conduct comprehensive experiments on OpenPSG dataset and achieve the state-of-art performance on the leadboard.
  arXiv  Detail & Related papers  (2023-02-06T09:47:46Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• Beyond the Prototype: Divide-and-conquer Proxies for Few-shot Segmentation [63.910211095033596]
  Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. We propose a simple yet versatile framework in the spirit of divide-and-conquer. Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
  arXiv  Detail & Related papers  (2022-04-21T06:21:14Z)
• CPSeg: Cluster-free Panoptic Segmentation of 3D LiDAR Point Clouds [2.891413712995641]
  We propose a novel real-time end-to-end panoptic segmentation network for LiDAR point clouds, called CPSeg. CPSeg comprises a shared encoder, a dual decoder, a task-aware attention module (TAM) and a cluster-free instance segmentation head.
  arXiv  Detail & Related papers  (2021-11-02T16:44:06Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Instance segmentation with the number of clusters incorporated in embedding learning [15.054120734581826]
  We propose to embed clustering information into an embedding learning framework FCRNet. FCRNet relieves the complexity of post process by incorporating the number of clustering groups into the embedding space. The superior performance of FCRNet is verified and compared with other methods on the nucleus dataset BBBC006.
  arXiv  Detail & Related papers  (2021-03-27T10:03:19Z)
• Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds [81.12016263972298]
  We present Panoster, a novel proposal-free panoptic segmentation method for LiDAR point clouds. Unlike previous approaches, Panoster proposes a simplified framework incorporating a learning-based clustering solution to identify instances. At inference time, this acts as a class-agnostic segmentation, allowing Panoster to be fast, while outperforming prior methods in terms of accuracy.
  arXiv  Detail & Related papers  (2020-10-28T18:10:20Z)
• SASO: Joint 3D Semantic-Instance Segmentation via Multi-scale Semantic Association and Salient Point Clustering Optimization [8.519716460338518]
  We propose a novel 3D point cloud segmentation framework named SASO, which jointly performs semantic and instance segmentation tasks. For semantic segmentation task, inspired by the inherent correlation among objects in spatial context, we propose a Multi-scale Semantic Association (MSA) module. For instance segmentation task, different from previous works that utilize clustering only in inference procedure, we propose a Salient Point Clustering Optimization (SPCO) module.
  arXiv  Detail & Related papers  (2020-06-25T08:55:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.