FPPN: Future Pseudo-LiDAR Frame Prediction for Autonomous Driving

• URL: http://arxiv.org/abs/2112.04401v1
• Date: Wed, 8 Dec 2021 16:46:18 GMT
• Title: FPPN: Future Pseudo-LiDAR Frame Prediction for Autonomous Driving
• Authors: Xudong Huang, Chunyu Lin, Haojie Liu, Lang Nie and Yao Zhao
• Abstract summary: We propose the first future pseudo-LiDAR frame prediction network. We first predict a future dense depth map based on dynamic motion information coarsely. We refine the predicted dense depth map using static contextual information. The future pseudo-LiDAR frame can be obtained by converting the predicted dense depth map into corresponding 3D point clouds.
• Score: 30.18167579599365
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LiDAR sensors are widely used in autonomous driving due to the reliable 3D spatial information. However, the data of LiDAR is sparse and the frequency of LiDAR is lower than that of cameras. To generate denser point clouds spatially and temporally, we propose the first future pseudo-LiDAR frame prediction network. Given the consecutive sparse depth maps and RGB images, we first predict a future dense depth map based on dynamic motion information coarsely. To eliminate the errors of optical flow estimation, an inter-frame aggregation module is proposed to fuse the warped depth maps with adaptive weights. Then, we refine the predicted dense depth map using static contextual information. The future pseudo-LiDAR frame can be obtained by converting the predicted dense depth map into corresponding 3D point clouds. Experimental results show that our method outperforms the existing solutions on the popular KITTI benchmark.

Related papers

• OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments [77.0399450848749]
  We propose an OccNeRF method for training occupancy networks without 3D supervision. We parameterize the reconstructed occupancy fields and reorganize the sampling strategy to align with the cameras' infinite perceptive range. For semantic occupancy prediction, we design several strategies to polish the prompts and filter the outputs of a pretrained open-vocabulary 2D segmentation model.
  arXiv  Detail & Related papers  (2023-12-14T18:58:52Z)
• Efficient Stereo Depth Estimation for Pseudo LiDAR: A Self-Supervised Approach Based on Multi-Input ResNet Encoder [0.0]
  This paper presents a strategy to obtain the real-time pseudo point cloud instead of the laser sensor from the image sensor. We propose an approach to use different depth estimators to obtain pseudo point clouds like LiDAR to obtain better performance.
  arXiv  Detail & Related papers  (2022-05-17T04:42:13Z)
• TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo [55.30992853477754]
  We present TANDEM, a real-time monocular tracking and dense framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of alignments. TANDEM shows state-of-the-art real-time 3D reconstruction performance.
  arXiv  Detail & Related papers  (2021-11-14T19:01:02Z)
• Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks [27.49539859498477]
  Exploiting past 3D LiDAR scans to predict future point clouds is a promising method for autonomous mobile systems. We propose an end-to-end approach that exploits a 2D range image representation of each 3D LiDAR scan. We develop an encoder-decoder architecture using 3D convolutions to jointly aggregate spatial and temporal information of the scene.
  arXiv  Detail & Related papers  (2021-09-28T19:58:13Z)
• Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR [22.202192422883122]
  We propose a novel two-stage network to advance the self-supervised monocular dense depth learning. Our model fuses monocular image features and sparse LiDAR features to predict initial depth maps. Our model outperforms the state-of-the-art sparse-LiDAR-based method (Pseudo-LiDAR++) by more than 68% for the downstream task monocular 3D object detection.
  arXiv  Detail & Related papers  (2021-09-20T15:28:36Z)
• Lifting 2D Object Locations to 3D by Discounting LiDAR Outliers across Objects and Views [70.1586005070678]
  We present a system for automatically converting 2D mask object predictions and raw LiDAR point clouds into full 3D bounding boxes of objects. Our method significantly outperforms previous work despite the fact that those methods use significantly more complex pipelines, 3D models and additional human-annotated external sources of prior information.
  arXiv  Detail & Related papers  (2021-09-16T13:01:13Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)
• SLPC: a VRNN-based approach for stochastic lidar prediction and completion in autonomous driving [63.87272273293804]
  We propose a new LiDAR prediction framework that is based on generative models namely Variational Recurrent Neural Networks (VRNNs) Our algorithm is able to address the limitations of previous video prediction frameworks when dealing with sparse data by spatially inpainting the depth maps in the upcoming frames. We present a sparse version of VRNNs and an effective self-supervised training method that does not require any labels.
  arXiv  Detail & Related papers  (2021-02-19T11:56:44Z)
• End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection [62.34374949726333]
  Pseudo-LiDAR (PL) has led to a drastic reduction in the accuracy gap between methods based on LiDAR sensors and those based on cheap stereo cameras. PL combines state-of-the-art deep neural networks for 3D depth estimation with those for 3D object detection by converting 2D depth map outputs to 3D point cloud inputs. We introduce a new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end.
  arXiv  Detail & Related papers  (2020-04-07T02:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.