End-To-End Optimization of LiDAR Beam Configuration for 3D Object
Detection and Localization
- URL: http://arxiv.org/abs/2201.03860v2
- Date: Tue, 28 Mar 2023 15:23:47 GMT
- Title: End-To-End Optimization of LiDAR Beam Configuration for 3D Object
Detection and Localization
- Authors: Niclas V\"odisch, Ozan Unal, Ke Li, Luc Van Gool, Dengxin Dai
- Abstract summary: We take a new route to learn to optimize the LiDAR beam configuration for a given application.
We propose a reinforcement learning-based learning-to-optimize framework to automatically optimize the beam configuration.
Our method is especially useful when a low-resolution (low-cost) LiDAR is needed.
- Score: 87.56144220508587
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Existing learning methods for LiDAR-based applications use 3D points scanned
under a pre-determined beam configuration, e.g., the elevation angles of beams
are often evenly distributed. Those fixed configurations are task-agnostic, so
simply using them can lead to sub-optimal performance. In this work, we take a
new route to learn to optimize the LiDAR beam configuration for a given
application. Specifically, we propose a reinforcement learning-based
learning-to-optimize (RL-L2O) framework to automatically optimize the beam
configuration in an end-to-end manner for different LiDAR-based applications.
The optimization is guided by the final performance of the target task and thus
our method can be integrated easily with any LiDAR-based application as a
simple drop-in module. The method is especially useful when a low-resolution
(low-cost) LiDAR is needed, for instance, for system deployment at a massive
scale. We use our method to search for the beam configuration of a
low-resolution LiDAR for two important tasks: 3D object detection and
localization. Experiments show that the proposed RL-L2O method improves the
performance in both tasks significantly compared to the baseline methods. We
believe that a combination of our method with the recent advances of
programmable LiDARs can start a new research direction for LiDAR-based active
perception. The code is publicly available at
https://github.com/vniclas/lidar_beam_selection
Related papers
- Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving [58.16024314532443]
We introduce LaserMix++, a framework that integrates laser beam manipulations from disparate LiDAR scans and incorporates LiDAR-camera correspondences to assist data-efficient learning.
Results demonstrate that LaserMix++ outperforms fully supervised alternatives, achieving comparable accuracy with five times fewer annotations.
This substantial advancement underscores the potential of semi-supervised approaches in reducing the reliance on extensive labeled data in LiDAR-based 3D scene understanding systems.
arXiv Detail & Related papers (2024-05-08T17:59:53Z) - Optimizing LiDAR Placements for Robust Driving Perception in Adverse Conditions [8.233185931617122]
The robustness of driving perception systems under unprecedented conditions is crucial for safety-critical usages.
We propose Place3D, a full-cycle pipeline that encompasses LiDAR placement optimization, data generation, and downstream evaluations.
We showcase exceptional robustness in both 3D object detection and LiDAR semantic segmentation tasks, under diverse adverse weather and sensor failure conditions.
arXiv Detail & Related papers (2024-03-25T17:59:58Z) - Improving Extrinsics between RADAR and LIDAR using Learning [18.211513930388417]
This paper presents a novel solution for 3D RADAR-LIDAR calibration in autonomous systems.
The method employs simple targets to generate data, including correspondence registration and a one-step optimization algorithm.
The proposed approach uses a deep learning framework such as PyTorch and can be optimized through gradient descent.
arXiv Detail & Related papers (2023-05-17T22:04:29Z) - Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving
Object Segmentation [23.666607237164186]
We propose a novel deep neural network exploiting both spatial-temporal information and different representation modalities of LiDAR scans to improve LiDAR-MOS performance.
Specifically, we first use a range image-based dual-branch structure to separately deal with spatial and temporal information.
We also use a point refinement module via 3D sparse convolution to fuse the information from both LiDAR range image and point cloud representations.
arXiv Detail & Related papers (2022-07-05T17:59:17Z) - Boosting 3D Object Detection by Simulating Multimodality on Point Clouds [51.87740119160152]
This paper presents a new approach to boost a single-modality (LiDAR) 3D object detector by teaching it to simulate features and responses that follow a multi-modality (LiDAR-image) detector.
The approach needs LiDAR-image data only when training the single-modality detector, and once well-trained, it only needs LiDAR data at inference.
Experimental results on the nuScenes dataset show that our approach outperforms all SOTA LiDAR-only 3D detectors.
arXiv Detail & Related papers (2022-06-30T01:44:30Z) - LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object
Detection [96.63947479020631]
In many real-world applications, the LiDAR points used by mass-produced robots and vehicles usually have fewer beams than that in large-scale public datasets.
We propose the LiDAR Distillation to bridge the domain gap induced by different LiDAR beams for 3D object detection.
arXiv Detail & Related papers (2022-03-28T17:59:02Z) - Learning Moving-Object Tracking with FMCW LiDAR [53.05551269151209]
We propose a learning-based moving-object tracking method utilizing our newly developed LiDAR sensor, Frequency Modulated Continuous Wave (FMCW) LiDAR.
Given the labels, we propose a contrastive learning framework, which pulls together the features from the same instance in embedding space and pushes apart the features from different instances to improve the tracking quality.
arXiv Detail & Related papers (2022-03-02T09:11:36Z) - SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural
Networks [81.64530401885476]
We propose a self-supervised LiDAR odometry method, dubbed SelfVoxeLO, to tackle these two difficulties.
Specifically, we propose a 3D convolution network to process the raw LiDAR data directly, which extracts features that better encode the 3D geometric patterns.
We evaluate our method's performances on two large-scale datasets, i.e., KITTI and Apollo-SouthBay.
arXiv Detail & Related papers (2020-10-19T09:23:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.