LiDAR-based 4D Occupancy Completion and Forecasting
- URL: http://arxiv.org/abs/2310.11239v1
- Date: Tue, 17 Oct 2023 13:08:24 GMT
- Title: LiDAR-based 4D Occupancy Completion and Forecasting
- Authors: Xinhao Liu, Moonjun Gong, Qi Fang, Haoyu Xie, Yiming Li, Hang Zhao,
Chen Feng
- Abstract summary: We introduce a novel LiDAR perception task of Occupancy Completion and Forecasting (OCF) in the context of autonomous driving.
This task requires new algorithms to address three challenges altogether: (1) sparse-to-dense reconstruction, (2) partial-to-complete hallucination, and (3) 3D-to-4D prediction.
We envision that this research will inspire and call for further investigation in this evolving and crucial area of 4D perception.
- Score: 36.655620377951024
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Scene completion and forecasting are two popular perception problems in
research for mobile agents like autonomous vehicles. Existing approaches treat
the two problems in isolation, resulting in a separate perception of the two
aspects. In this paper, we introduce a novel LiDAR perception task of Occupancy
Completion and Forecasting (OCF) in the context of autonomous driving to unify
these aspects into a cohesive framework. This task requires new algorithms to
address three challenges altogether: (1) sparse-to-dense reconstruction, (2)
partial-to-complete hallucination, and (3) 3D-to-4D prediction. To enable
supervision and evaluation, we curate a large-scale dataset termed OCFBench
from public autonomous driving datasets. We analyze the performance of closely
related existing baseline models and our own ones on our dataset. We envision
that this research will inspire and call for further investigation in this
evolving and crucial area of 4D perception. Our code for data curation and
baseline implementation is available at https://github.com/ai4ce/Occ4cast.
Related papers
- Scalable Vision-Based 3D Object Detection and Monocular Depth Estimation
for Autonomous Driving [5.347428263669927]
This dissertation is a multifaceted contribution to the advancement of vision-based 3D perception technologies.
In the first segment, the thesis introduces structural enhancements to both monocular and stereo 3D object detection algorithms.
The second segment is devoted to data-driven strategies and their real-world applications in 3D vision detection.
arXiv Detail & Related papers (2024-03-04T13:42:54Z) - Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting [58.45661235893729]
One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences.
We show that this task requires algorithms to implicitly capture (1) sensor extrinsics (i.e., the egomotion of the autonomous vehicle), (2) sensor intrinsics (i.e., the sampling pattern specific to the particular LiDAR sensor), and (3) the shape and motion of other objects in the scene.
We render point cloud data from 4D occupancy predictions given sensor extrinsics and intrinsics, allowing one to train and test occupancy algorithms with unannotated LiDAR sequences.
arXiv Detail & Related papers (2023-02-25T18:12:37Z) - GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D
LiDAR Segmentation [60.07812405063708]
3D point cloud semantic segmentation is fundamental for autonomous driving.
Most approaches in the literature neglect an important aspect, i.e., how to deal with domain shift when handling dynamic scenes.
This paper advances the state of the art in this research field.
arXiv Detail & Related papers (2022-07-20T09:06:07Z) - DOLPHINS: Dataset for Collaborative Perception enabled Harmonious and
Interconnected Self-driving [19.66714697653504]
Vehicle-to-Everything (V2X) network has enabled collaborative perception in autonomous driving.
The lack of datasets has severely blocked the development of collaborative perception algorithms.
We release DOLPHINS: dataset for cOllaborative Perception enabled Harmonious and INterconnected Self-driving.
arXiv Detail & Related papers (2022-07-15T17:07:07Z) - Learning-based Point Cloud Registration for 6D Object Pose Estimation in
the Real World [55.7340077183072]
We tackle the task of estimating the 6D pose of an object from point cloud data.
Recent learning-based approaches to addressing this task have shown great success on synthetic datasets.
We analyze the causes of these failures, which we trace back to the difference between the feature distributions of the source and target point clouds.
arXiv Detail & Related papers (2022-03-29T07:55:04Z) - LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm.
Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods in both tasks.
We extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.
arXiv Detail & Related papers (2022-03-14T15:25:42Z) - Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data
Augmentation [77.60050239225086]
We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images.
Our approach is fully automatic without any human interaction.
We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
arXiv Detail & Related papers (2020-12-15T03:03:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.