LiDAR-based 4D Occupancy Completion and Forecasting

• URL: http://arxiv.org/abs/2310.11239v1
• Date: Tue, 17 Oct 2023 13:08:24 GMT
• Title: LiDAR-based 4D Occupancy Completion and Forecasting
• Authors: Xinhao Liu, Moonjun Gong, Qi Fang, Haoyu Xie, Yiming Li, Hang Zhao, Chen Feng
• Abstract summary: We introduce a novel LiDAR perception task of Occupancy Completion and Forecasting (OCF) in the context of autonomous driving. This task requires new algorithms to address three challenges altogether: (1) sparse-to-dense reconstruction, (2) partial-to-complete hallucination, and (3) 3D-to-4D prediction. We envision that this research will inspire and call for further investigation in this evolving and crucial area of 4D perception.
• Score: 36.655620377951024
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Scene completion and forecasting are two popular perception problems in research for mobile agents like autonomous vehicles. Existing approaches treat the two problems in isolation, resulting in a separate perception of the two aspects. In this paper, we introduce a novel LiDAR perception task of Occupancy Completion and Forecasting (OCF) in the context of autonomous driving to unify these aspects into a cohesive framework. This task requires new algorithms to address three challenges altogether: (1) sparse-to-dense reconstruction, (2) partial-to-complete hallucination, and (3) 3D-to-4D prediction. To enable supervision and evaluation, we curate a large-scale dataset termed OCFBench from public autonomous driving datasets. We analyze the performance of closely related existing baseline models and our own ones on our dataset. We envision that this research will inspire and call for further investigation in this evolving and crucial area of 4D perception. Our code for data curation and baseline implementation is available at https://github.com/ai4ce/Occ4cast.

Related papers

• Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving [14.403130104985557]
  This paper presents a novel dataset for anomaly segmentation in driving scenarios.<n>It is the first publicly available dataset focused on road anomaly segmentation with dense 3D semantic labeling.<n>Our dataset and evaluation code will be openly available, facilitating the testing and performance comparison of different approaches.
  arXiv  Detail & Related papers  (2025-05-04T15:15:35Z)
• InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios [13.821143687548494]
  This paper introduces a new 3D infrastructure-side collaborative perception dataset, abbreviated as inscope. InScope encapsulates a 20-day capture duration with 303 tracking trajectories and 187,787 3D bounding boxes annotated by experts.
  arXiv  Detail & Related papers  (2024-07-31T13:11:14Z)
• Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting [58.45661235893729]
  One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences. We show that this task requires algorithms to implicitly capture (1) sensor extrinsics (i.e., the egomotion of the autonomous vehicle), (2) sensor intrinsics (i.e., the sampling pattern specific to the particular LiDAR sensor), and (3) the shape and motion of other objects in the scene. We render point cloud data from 4D occupancy predictions given sensor extrinsics and intrinsics, allowing one to train and test occupancy algorithms with unannotated LiDAR sequences.
  arXiv  Detail & Related papers  (2023-02-25T18:12:37Z)
• GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation [60.07812405063708]
  3D point cloud semantic segmentation is fundamental for autonomous driving. Most approaches in the literature neglect an important aspect, i.e., how to deal with domain shift when handling dynamic scenes. This paper advances the state of the art in this research field.
  arXiv  Detail & Related papers  (2022-07-20T09:06:07Z)
• DOLPHINS: Dataset for Collaborative Perception enabled Harmonious and Interconnected Self-driving [19.66714697653504]
  Vehicle-to-Everything (V2X) network has enabled collaborative perception in autonomous driving. The lack of datasets has severely blocked the development of collaborative perception algorithms. We release DOLPHINS: dataset for cOllaborative Perception enabled Harmonious and INterconnected Self-driving.
  arXiv  Detail & Related papers  (2022-07-15T17:07:07Z)
• Learning-based Point Cloud Registration for 6D Object Pose Estimation in the Real World [55.7340077183072]
  We tackle the task of estimating the 6D pose of an object from point cloud data. Recent learning-based approaches to addressing this task have shown great success on synthetic datasets. We analyze the causes of these failures, which we trace back to the difference between the feature distributions of the source and target point clouds.
  arXiv  Detail & Related papers  (2022-03-29T07:55:04Z)
• LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods in both tasks. We extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.
  arXiv  Detail & Related papers  (2022-03-14T15:25:42Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  LiDAR-based panoptic segmentation aims to parse both objects and scenes in a unified manner. We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-24T08:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.