LAPTNet: LiDAR-Aided Perspective Transform Network

• URL: http://arxiv.org/abs/2211.14445v1
• Date: Mon, 14 Nov 2022 18:56:02 GMT
• Title: LAPTNet: LiDAR-Aided Perspective Transform Network
• Authors: Manuel Alejandro Diaz-Zapata (CHROMA), \"Ozg\"ur Erkent (CHROMA), Christian Laugier (CHROMA), Jilles Dibangoye (CHROMA), David Sierra Gonz\'alez (CHROMA)
• Abstract summary: We present an architecture that fuses LiDAR and camera information to generate semantic grids. LAPTNet is able to associate features in the camera plane to the bird's eye view without having to predict any depth information about the scene.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic grids are a useful representation of the environment around a robot. They can be used in autonomous vehicles to concisely represent the scene around the car, capturing vital information for downstream tasks like navigation or collision assessment. Information from different sensors can be used to generate these grids. Some methods rely only on RGB images, whereas others choose to incorporate information from other sensors, such as radar or LiDAR. In this paper, we present an architecture that fuses LiDAR and camera information to generate semantic grids. By using the 3D information from a LiDAR point cloud, the LiDAR-Aided Perspective Transform Network (LAPTNet) is able to associate features in the camera plane to the bird's eye view without having to predict any depth information about the scene. Compared to state-of-theart camera-only methods, LAPTNet achieves an improvement of up to 8.8 points (or 38.13%) over state-of-art competing approaches for the classes proposed in the NuScenes dataset validation split.

Related papers

• HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation [106.09886920774002]
  We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network. Our method achieves consistent improvements over the baseline trained from scratch and significantly out- performs the existing schemes.
  arXiv  Detail & Related papers  (2024-03-18T14:18:08Z)
• Vision meets mmWave Radar: 3D Object Perception Benchmark for Autonomous Driving [30.456314610767667]
  We introduce the CRUW3D dataset, including 66K synchronized and well-calibrated camera, radar, and LiDAR frames. This kind of format can enable machine learning models to more reliable perception results after fusing the information or features between the camera and radar.
  arXiv  Detail & Related papers  (2023-11-17T01:07:37Z)
• MaskedFusion360: Reconstruct LiDAR Data by Querying Camera Features [11.28654979274464]
  In self-driving applications, LiDAR data provides accurate information about distances in 3D but lacks the semantic richness of camera data. We introduce a novel self-supervised method to fuse LiDAR and camera data for self-driving applications.
  arXiv  Detail & Related papers  (2023-06-12T13:01:33Z)
• LAPTNet-FPN: Multi-scale LiDAR-aided Projective Transform Network for Real Time Semantic Grid Prediction [0.0]
  By fusing information from multiple sensors, robustness can be increased and the computational load for the task can be lowered. Our multi-scale LiDAR-Aided Perspective Transform network uses information available in point clouds to guide the projection of image features to a top-view representation.
  arXiv  Detail & Related papers  (2023-02-10T12:34:28Z)
• Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds [94.21415132135951]
  We propose to detect 3D objects by exploiting temporal information in multiple frames. We implement our algorithm based on prevalent anchor-based and anchor-free detectors.
  arXiv  Detail & Related papers  (2022-07-26T05:16:28Z)
• Paint and Distill: Boosting 3D Object Detection with Semantic Passing Network [70.53093934205057]
  3D object detection task from lidar or camera sensors is essential for autonomous driving. We propose a novel semantic passing framework, named SPNet, to boost the performance of existing lidar-based 3D detection models.
  arXiv  Detail & Related papers  (2022-07-12T12:35:34Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• PC-DAN: Point Cloud based Deep Affinity Network for 3D Multi-Object Tracking (Accepted as an extended abstract in JRDB-ACT Workshop at CVPR21) [68.12101204123422]
  A point cloud is a dense compilation of spatial data in 3D coordinates. We propose a PointNet-based approach for 3D Multi-Object Tracking (MOT)
  arXiv  Detail & Related papers  (2021-06-03T05:36:39Z)
• End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection [62.34374949726333]
  Pseudo-LiDAR (PL) has led to a drastic reduction in the accuracy gap between methods based on LiDAR sensors and those based on cheap stereo cameras. PL combines state-of-the-art deep neural networks for 3D depth estimation with those for 3D object detection by converting 2D depth map outputs to 3D point cloud inputs. We introduce a new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end.
  arXiv  Detail & Related papers  (2020-04-07T02:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.