Towards Robust 3D Object Detection In Rainy Conditions
- URL: http://arxiv.org/abs/2310.00944v2
- Date: Thu, 5 Oct 2023 06:37:23 GMT
- Title: Towards Robust 3D Object Detection In Rainy Conditions
- Authors: Aldi Piroli, Vinzenz Dallabetta, Johannes Kopp, Marc Walessa, Daniel
Meissner, Klaus Dietmayer
- Abstract summary: We propose a framework for improving the robustness of LiDAR-based 3D object detectors against road spray.
Our approach uses a state-of-the-art adverse weather detection network to filter out spray from the LiDAR point cloud.
In addition to adverse weather filtering, we explore the use of radar targets to further filter false positive detections.
- Score: 10.920640666237833
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: LiDAR sensors are used in autonomous driving applications to accurately
perceive the environment. However, they are affected by adverse weather
conditions such as snow, fog, and rain. These everyday phenomena introduce
unwanted noise into the measurements, severely degrading the performance of
LiDAR-based perception systems. In this work, we propose a framework for
improving the robustness of LiDAR-based 3D object detectors against road spray.
Our approach uses a state-of-the-art adverse weather detection network to
filter out spray from the LiDAR point cloud, which is then used as input for
the object detector. In this way, the detected objects are less affected by the
adverse weather in the scene, resulting in a more accurate perception of the
environment. In addition to adverse weather filtering, we explore the use of
radar targets to further filter false positive detections. Tests on real-world
data show that our approach improves the robustness to road spray of several
popular 3D object detectors.
Related papers
- Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers.
Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements.
We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
arXiv Detail & Related papers (2024-05-07T20:44:48Z) - Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust
3D Object Detection [26.278415287992964]
Previous research has attempted to address this by simulating the noise from rain to improve the robustness of detection models.
We propose a novel rain simulation method, termed DRET, that unifies Dynamics and Rainy Environment Theory.
We also present a Sunny-to-Rainy Knowledge Distillation approach to enhance 3D detection under rainy conditions.
arXiv Detail & Related papers (2024-02-28T17:21:02Z) - Energy-based Detection of Adverse Weather Effects in LiDAR Data [7.924836086640871]
We propose a novel approach for detecting adverse weather effects in LiDAR data.
Our method learns to associate low energy scores with inlier points and high energy scores with outliers.
To help expand the research field of LiDAR perception in adverse weather, we release the SemanticSpray dataset.
arXiv Detail & Related papers (2023-05-25T15:03:36Z) - Survey on LiDAR Perception in Adverse Weather Conditions [6.317642241067219]
The active LiDAR sensor is able to create an accurate 3D representation of a scene.
The LiDAR's performance change under adverse weather conditions like fog, snow or rain.
We address topics such as the availability of appropriate data, raw point cloud processing and denoising, robust perception algorithms and sensor fusion to mitigate adverse weather induced shortcomings.
arXiv Detail & Related papers (2023-04-13T07:45:23Z) - LiDAR Snowfall Simulation for Robust 3D Object Detection [116.10039516404743]
We propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds.
Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the measurement for each LiDAR beam.
We use our simulation to generate partially synthetic snowy LiDAR data and leverage these data for training 3D object detection models that are robust to snowfall.
arXiv Detail & Related papers (2022-03-28T21:48:26Z) - Anomaly Detection in Radar Data Using PointNets [7.3600716208089825]
We present an approach based on PointNets to detect anomalous radar targets.
Our method is evaluated on a real-world dataset in urban scenarios.
arXiv Detail & Related papers (2021-09-20T10:02:24Z) - Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in
Adverse Weather [92.84066576636914]
This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather.
We tackle this problem by simulating physically accurate fog into clear-weather scenes.
We are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset.
arXiv Detail & Related papers (2021-08-11T14:37:54Z) - Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of
Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars.
They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
arXiv Detail & Related papers (2021-07-14T21:10:47Z) - Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object
Detection [55.12894776039135]
State-of-the-art 3D object detectors, based on deep learning, have shown promising accuracy but are prone to over-fit to domain idiosyncrasies.
We propose a novel learning approach that drastically reduces this gap by fine-tuning the detector on pseudo-labels in the target domain.
We show, on five autonomous driving datasets, that fine-tuning the detector on these pseudo-labels substantially reduces the domain gap to new driving environments.
arXiv Detail & Related papers (2021-03-26T01:18:11Z) - Physically Realizable Adversarial Examples for LiDAR Object Detection [72.0017682322147]
We present a method to generate universal 3D adversarial objects to fool LiDAR detectors.
In particular, we demonstrate that placing an adversarial object on the rooftop of any target vehicle to hide the vehicle entirely from LiDAR detectors with a success rate of 80%.
This is one step closer towards safer self-driving under unseen conditions from limited training data.
arXiv Detail & Related papers (2020-04-01T16:11:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.