Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking

• URL: http://arxiv.org/abs/2109.03805v2
• Date: Fri, 10 Sep 2021 05:10:11 GMT
• Title: Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking
• Authors: Whye Kit Fong, Rohit Mohan, Juana Valeria Hurtado, Lubing Zhou, Holger Caesar, Oscar Beijbom, and Abhinav Valada
• Abstract summary: We introduce the large-scale Panoptic nuScenes benchmark dataset that extends our popular nuScenes dataset. We analyze the drawbacks of the existing metrics for panoptic tracking and propose the novel instance-centric PAT metric. We believe that this extension will accelerate the research of novel methods for scene understanding of dynamic urban environments.
• Score: 11.950994311766898
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Panoptic scene understanding and tracking of dynamic agents are essential for robots and automated vehicles to navigate in urban environments. As LiDARs provide accurate illumination-independent geometric depictions of the scene, performing these tasks using LiDAR point clouds provides reliable predictions. However, existing datasets lack diversity in the type of urban scenes and have a limited number of dynamic object instances which hinders both learning of these tasks as well as credible benchmarking of the developed methods. In this paper, we introduce the large-scale Panoptic nuScenes benchmark dataset that extends our popular nuScenes dataset with point-wise groundtruth annotations for semantic segmentation, panoptic segmentation, and panoptic tracking tasks. To facilitate comparison, we provide several strong baselines for each of these tasks on our proposed dataset. Moreover, we analyze the drawbacks of the existing metrics for panoptic tracking and propose the novel instance-centric PAT metric that addresses the concerns. We present exhaustive experiments that demonstrate the utility of Panoptic nuScenes compared to existing datasets and make the online evaluation server available at nuScenes.org. We believe that this extension will accelerate the research of novel methods for scene understanding of dynamic urban environments.

Related papers

• OmniPose6D: Towards Short-Term Object Pose Tracking in Dynamic Scenes from Monocular RGB [40.62577054196799]
  We introduce a large-scale synthetic dataset OmniPose6D, crafted to mirror the diversity of real-world conditions. We present a benchmarking framework for a comprehensive comparison of pose tracking algorithms.
  arXiv  Detail & Related papers  (2024-10-09T09:01:40Z)
• LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry [52.131996528655094]
  We present the Long-term Effective Any Point Tracking (LEAP) module. LEAP innovatively combines visual, inter-track, and temporal cues with mindfully selected anchors for dynamic track estimation. Based on these traits, we develop LEAP-VO, a robust visual odometry system adept at handling occlusions and dynamic scenes.
  arXiv  Detail & Related papers  (2024-01-03T18:57:27Z)
• OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments [77.0399450848749]
  We propose an OccNeRF method for training occupancy networks without 3D supervision. We parameterize the reconstructed occupancy fields and reorganize the sampling strategy to align with the cameras' infinite perceptive range. For semantic occupancy prediction, we design several strategies to polish the prompts and filter the outputs of a pretrained open-vocabulary 2D segmentation model.
  arXiv  Detail & Related papers  (2023-12-14T18:58:52Z)
• MemorySeg: Online LiDAR Semantic Segmentation with a Latent Memory [43.47217183838879]
  We propose a novel framework for semantic segmentation of temporal sequence of LiDAR point clouds. We use a memory network to store, update and retrieve past information. Our framework also includes a regularizer that penalizes prediction variations in the neighborhood of the point cloud.
  arXiv  Detail & Related papers  (2023-11-02T19:18:34Z)
• LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting [65.71129509623587]
  Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning. However, the promising results achieved on current public datasets may not be applicable to practical scenarios. We introduce the LargeST benchmark dataset, which includes a total of 8,600 sensors in California with a 5-year time coverage.
  arXiv  Detail & Related papers  (2023-06-14T05:48:36Z)
• DOORS: Dataset fOr bOuldeRs Segmentation. Statistical properties and Blender setup [0.0]
  The capability to detect boulders on the surface of small bodies is beneficial for vision-based applications such as hazard detection during critical operations and navigation. This work provides a statistical characterization and setup used for the generation of two datasets about boulders on small bodies that are made publicly available.
  arXiv  Detail & Related papers  (2022-10-28T16:39:06Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• Deep residential representations: Using unsupervised learning to unlock elevation data for geo-demographic prediction [0.0]
  LiDAR technology can be used to provide detailed three-dimensional elevation maps of urban and rural landscapes. To date, airborne LiDAR imaging has been predominantly confined to the environmental and archaeological domains. We consider the suitability of this data not just on its own but also as a source of data in combination with demographic features, thus providing a realistic use case for the embeddings.
  arXiv  Detail & Related papers  (2021-12-02T17:10:52Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Visual Tracking by TridentAlign and Context Embedding [71.60159881028432]
  We propose novel TridentAlign and context embedding modules for Siamese network-based visual tracking methods. The performance of the proposed tracker is comparable to that of state-of-the-art trackers, while the proposed tracker runs at real-time speed.
  arXiv  Detail & Related papers  (2020-07-14T08:00:26Z)
• MOPT: Multi-Object Panoptic Tracking [33.77171216778909]
  We introduce a novel perception task denoted as multi-object panoptic tracking (MOPT) MOPT allows for exploiting pixel-level semantic information of 'thing' and'stuff' classes, temporal coherence, and pixel-level associations over time. We present extensive quantitative and qualitative evaluations of both vision-based and LiDAR-based MOPT that demonstrate encouraging results.
  arXiv  Detail & Related papers  (2020-04-17T11:45:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.