Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
- URL: http://arxiv.org/abs/2103.14962v1
- Date: Sat, 27 Mar 2021 18:31:40 GMT
- Title: Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
- Authors: Zixiang Zhou, Yang Zhang, Hassan Foroosh
- Abstract summary: We present a fast and robust LiDAR point cloud panoptic segmentation framework, referred to as Panoptic-PolarNet.
We learn both semantic segmentation and class-agnostic instance clustering in a single inference network using a polar Bird's Eye View representation.
Our experiments show that Panoptic-PolarNet outperforms the baseline methods on Semantic KITTI and nuScenes datasets with an almost real-time inference speed.
- Score: 21.296942497092402
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Panoptic segmentation presents a new challenge in exploiting the merits of
both detection and segmentation, with the aim of unifying instance segmentation
and semantic segmentation in a single framework. However, an efficient solution
for panoptic segmentation in the emerging domain of LiDAR point cloud is still
an open research problem and is very much under-explored. In this paper, we
present a fast and robust LiDAR point cloud panoptic segmentation framework,
referred to as Panoptic-PolarNet. We learn both semantic segmentation and
class-agnostic instance clustering in a single inference network using a polar
Bird's Eye View (BEV) representation, enabling us to circumvent the issue of
occlusion among instances in urban street scenes. To improve our network's
learnability, we also propose an adapted instance augmentation technique and a
novel adversarial point cloud pruning method. Our experiments show that
Panoptic-PolarNet outperforms the baseline methods on SemanticKITTI and
nuScenes datasets with an almost real-time inference speed. Panoptic-PolarNet
achieved 54.1% PQ in the public SemanticKITTI panoptic segmentation leaderboard
and leading performance for the validation set of nuScenes.
Related papers
- Lidar Panoptic Segmentation and Tracking without Bells and Whistles [48.078270195629415]
We propose a detection-centric network for lidar segmentation and tracking.
One of the core components of our network is the object instance detection branch.
We evaluate our method on several 3D/4D LPS benchmarks and observe that our model establishes a new state-of-the-art among open-sourced models.
arXiv Detail & Related papers (2023-10-19T04:44:43Z) - Pointly-Supervised Panoptic Segmentation [106.68888377104886]
We propose a new approach to applying point-level annotations for weakly-supervised panoptic segmentation.
Instead of the dense pixel-level labels used by fully supervised methods, point-level labels only provide a single point for each target as supervision.
We formulate the problem in an end-to-end framework by simultaneously generating panoptic pseudo-masks from point-level labels and learning from them.
arXiv Detail & Related papers (2022-10-25T12:03:51Z) - GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation [91.15865862160088]
We introduce a geometric flow network (GFNet) to explore the geometric correspondence between different views in an align-before-fuse manner.
Specifically, we devise a novel geometric flow module (GFM) to bidirectionally align and propagate the complementary information across different views.
arXiv Detail & Related papers (2022-07-06T11:48:08Z) - Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic
Segmentation via Clustering Pseudo Heatmap [9.770808277353128]
We propose a fast and high-performance LiDAR-based framework, referred to as Panoptic-PHNet.
We introduce a clustering pseudo heatmap as a new paradigm, which, followed by a center grouping module, yields instance centers for efficient clustering.
For backbone design, we fuse the fine-grained voxel features and the 2D Bird's Eye View (BEV) features with different receptive fields to utilize both detailed and global information.
arXiv Detail & Related papers (2022-05-14T08:16:13Z) - CPSeg: Cluster-free Panoptic Segmentation of 3D LiDAR Point Clouds [2.891413712995641]
We propose a novel real-time end-to-end panoptic segmentation network for LiDAR point clouds, called CPSeg.
CPSeg comprises a shared encoder, a dual decoder, a task-aware attention module (TAM) and a cluster-free instance segmentation head.
arXiv Detail & Related papers (2021-11-02T16:44:06Z) - SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional
Attention Clustering [1.1470070927586016]
We present a learnable sparse multi-directional attention clustering to segment multi-scale foreground instances.
SMAC-Seg is a real-time clustering-based approach, which removes the complex proposal network to segment instances.
Our experimental results show that SMAC-Seg achieves state-of-the-art performance among all real-time deployable networks.
arXiv Detail & Related papers (2021-08-31T02:25:01Z) - Fully Convolutional Networks for Panoptic Segmentation with Point-based
Supervision [88.71403886207071]
We present a conceptually simple, strong, and efficient framework for fully- and weakly-supervised panoptic segmentation, called Panoptic FCN.
Our approach aims to represent and predict foreground things and background stuff in a unified fully convolutional pipeline.
Panoptic FCN encodes each object instance or stuff category with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly.
arXiv Detail & Related papers (2021-08-17T15:28:53Z) - Exemplar-Based Open-Set Panoptic Segmentation Network [79.99748041746592]
We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task.
We investigate the practical challenges of the task and construct a benchmark on top of an existing dataset, COCO.
We propose a novel exemplar-based open-set panoptic segmentation network (EOPSN) inspired by exemplar theory.
arXiv Detail & Related papers (2021-05-18T07:59:21Z) - Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds [81.12016263972298]
We present Panoster, a novel proposal-free panoptic segmentation method for LiDAR point clouds.
Unlike previous approaches, Panoster proposes a simplified framework incorporating a learning-based clustering solution to identify instances.
At inference time, this acts as a class-agnostic segmentation, allowing Panoster to be fast, while outperforming prior methods in terms of accuracy.
arXiv Detail & Related papers (2020-10-28T18:10:20Z) - Towards Bounding-Box Free Panoptic Segmentation [16.4548904544277]
We introduce a new Bounding-Box Free Network (BBFNet) for panoptic segmentation.
BBFNet predicts coarse watershed levels and uses them to detect large instance candidates where boundaries are well defined.
For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects.
arXiv Detail & Related papers (2020-02-18T16:34:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.