CPSeg: Cluster-free Panoptic Segmentation of 3D LiDAR Point Clouds

• URL: http://arxiv.org/abs/2111.01723v1
• Date: Tue, 2 Nov 2021 16:44:06 GMT
• Title: CPSeg: Cluster-free Panoptic Segmentation of 3D LiDAR Point Clouds
• Authors: Enxu Li, Ryan Razani, Yixuan Xu, Bingbing Liu
• Abstract summary: We propose a novel real-time end-to-end panoptic segmentation network for LiDAR point clouds, called CPSeg. CPSeg comprises a shared encoder, a dual decoder, a task-aware attention module (TAM) and a cluster-free instance segmentation head.
• Score: 2.891413712995641
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: A fast and accurate panoptic segmentation system for LiDAR point clouds is crucial for autonomous driving vehicles to understand the surrounding objects and scenes. Existing approaches usually rely on proposals or clustering to segment foreground instances. As a result, they struggle to achieve real-time performance. In this paper, we propose a novel real-time end-to-end panoptic segmentation network for LiDAR point clouds, called CPSeg. In particular, CPSeg comprises a shared encoder, a dual decoder, a task-aware attention module (TAM) and a cluster-free instance segmentation head. TAM is designed to enforce these two decoders to learn rich task-aware features for semantic and instance embedding. Moreover, CPSeg incorporates a new cluster-free instance segmentation head to dynamically pillarize foreground points according to the learned embedding. Then, it acquires instance labels by finding connected pillars with a pairwise embedding comparison. Thus, the conventional proposal-based or clustering-based instance segmentation is transformed into a binary segmentation problem on the pairwise embedding comparison matrix. To help the network regress instance embedding, a fast and deterministic depth completion algorithm is proposed to calculate surface normal of each point cloud in real-time. The proposed method is benchmarked on two large-scale autonomous driving datasets, namely, SemanticKITTI and nuScenes. Notably, extensive experimental results show that CPSeg achieves the state-of-the-art results among real-time approaches on both datasets.

Related papers

• Lidar Panoptic Segmentation and Tracking without Bells and Whistles [48.078270195629415]
  We propose a detection-centric network for lidar segmentation and tracking. One of the core components of our network is the object instance detection branch. We evaluate our method on several 3D/4D LPS benchmarks and observe that our model establishes a new state-of-the-art among open-sourced models.
  arXiv  Detail & Related papers  (2023-10-19T04:44:43Z)
• PANet: LiDAR Panoptic Segmentation with Sparse Instance Proposal and Aggregation [15.664835767712775]
  This work proposes a new LPS framework named PANet to eliminate the dependency on the offset branch. PaNet achieves state-of-the-art performance among published works on the Semantic KITII validation and nuScenes validation for the panoptic segmentation task.
  arXiv  Detail & Related papers  (2023-06-27T10:02:28Z)
• PUPS: Point Cloud Unified Panoptic Segmentation [13.668363631123649]
  We propose a simple but effective point cloud unified panoptic segmentation (PUPS) framework. PUPS uses a set of point-level classifiers to directly predict semantic and instance groupings in an end-to-end manner. PUPS achieves 1st place on the leader board of Semantic KITTI panoptic segmentation task and state-of-the-art results on nuScenes.
  arXiv  Detail & Related papers  (2023-02-13T08:42:41Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Sparse Instance Activation for Real-Time Instance Segmentation [72.23597664935684]
  We propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation. SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark.
  arXiv  Detail & Related papers  (2022-03-24T03:15:39Z)
• Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining [17.69745159912481]
  We introduce the first semi-supervised point cloud instance segmentation framework (SPIB) using both labeled and unlabelled bounding boxes as supervision. Our method can achieve competitive performance compared with the recent fully-supervised methods.
  arXiv  Detail & Related papers  (2021-11-30T08:40:40Z)
• SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering [1.1470070927586016]
  We present a learnable sparse multi-directional attention clustering to segment multi-scale foreground instances. SMAC-Seg is a real-time clustering-based approach, which removes the complex proposal network to segment instances. Our experimental results show that SMAC-Seg achieves state-of-the-art performance among all real-time deployable networks.
  arXiv  Detail & Related papers  (2021-08-31T02:25:01Z)
• SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization [52.20602782690776]
  It is expensive and tedious to obtain large scale paired sparse-canned point sets for training from real scanned sparse data. We propose a self-supervised point cloud upsampling network, named SPU-Net, to capture the inherent upsampling patterns of points lying on the underlying object surface. We conduct various experiments on both synthetic and real-scanned datasets, and the results demonstrate that we achieve comparable performance to the state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2020-12-08T14:14:09Z)
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  LiDAR-based panoptic segmentation aims to parse both objects and scenes in a unified manner. We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-24T08:44:46Z)
• Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
  We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method. Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
  arXiv  Detail & Related papers  (2020-06-22T08:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.