Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection

• URL: http://arxiv.org/abs/2107.07004v1
• Date: Wed, 14 Jul 2021 21:10:47 GMT
• Title: Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection
• Authors: Velat Kilic, Deepti Hegde, Vishwanath Sindagi, A. Brinton Cooper, Mark A. Foster and Vishal M. Patel
• Abstract summary: Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
• Score: 60.89616629421904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. However, they are known to be sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR). As a result, lidar-based object detectors trained on data captured in normal weather tend to perform poorly in such scenarios. However, collecting and labelling sufficient training data in a diverse range of adverse weather conditions is laborious and prohibitively expensive. To address this issue, we propose a physics-based approach to simulate lidar point clouds of scenes in adverse weather conditions. These augmented datasets can then be used to train lidar-based detectors to improve their all-weather reliability. Specifically, we introduce a hybrid Monte-Carlo based approach that treats (i) the effects of large particles by placing them randomly and comparing their back reflected power against the target, and (ii) attenuation effects on average through calculation of scattering efficiencies from the Mie theory and particle size distributions. Retraining networks with this augmented data improves mean average precision evaluated on real world rainy scenes and we observe greater improvement in performance with our model relative to existing models from the literature. Furthermore, we evaluate recent state-of-the-art detectors on the simulated weather conditions and present an in-depth analysis of their performance.

Related papers

• Robust ADAS: Enhancing Robustness of Machine Learning-based Advanced Driver Assistance Systems for Adverse Weather [5.383130566626935]
  This paper employs a Denoising Deep Neural Network as a preprocessing step to transform adverse weather images into clear weather images. It improves driver visualization, which is critical for safe navigation in adverse weather conditions.
  arXiv  Detail & Related papers  (2024-07-02T18:03:52Z)
• Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time [1.1725016312484975]
  Lidar-based object detection is significantly affected by adverse weather conditions such as rain and fog. Our research provides a comprehensive study of effective methods for mitigating the effects of adverse weather on the reliability of lidar-based object detection.
  arXiv  Detail & Related papers  (2024-01-17T08:31:58Z)
• Realistic Rainy Weather Simulation for LiDARs in CARLA Simulator [17.605152926277707]
  We propose a simulator-based physical modeling approach to augment LiDAR data in rainy weather. We complete the modeling task of the rainy weather in the CARLA simulator and establish a pipeline for LiDAR data collection. In the experiment, we observe that the model augmented by the synthetic data improves the object detection task's performance.
  arXiv  Detail & Related papers  (2023-12-20T05:16:04Z)
• Energy-based Detection of Adverse Weather Effects in LiDAR Data [7.924836086640871]
  We propose a novel approach for detecting adverse weather effects in LiDAR data. Our method learns to associate low energy scores with inlier points and high energy scores with outliers. To help expand the research field of LiDAR perception in adverse weather, we release the SemanticSpray dataset.
  arXiv  Detail & Related papers  (2023-05-25T15:03:36Z)
• Learning to Simulate Realistic LiDARs [66.7519667383175]
  We introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces. We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly.
  arXiv  Detail & Related papers  (2022-09-22T13:12:54Z)
• Robust 3D Object Detection in Cold Weather Conditions [7.924836086640871]
  Adverse weather conditions can negatively affect LiDAR-based object detectors. We focus on the phenomenon of vehicle gas exhaust condensation in cold weather conditions. We propose to solve this problem by using data augmentation and a novel training loss term.
  arXiv  Detail & Related papers  (2022-05-24T09:37:07Z)
• LiDAR Snowfall Simulation for Robust 3D Object Detection [116.10039516404743]
  We propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds. Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the measurement for each LiDAR beam. We use our simulation to generate partially synthetic snowy LiDAR data and leverage these data for training 3D object detection models that are robust to snowfall.
  arXiv  Detail & Related papers  (2022-03-28T21:48:26Z)
• Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing [70.16043883381677]
  In autonomous racing, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres. In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions. We introduce an approach of using synthesised adverse condition datasets in autonomous racing (generated using CycleGAN) to improve the performance of four out of five state-of-the-art detectors.
  arXiv  Detail & Related papers  (2022-01-10T10:02:40Z)
• Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather [92.84066576636914]
  This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. We tackle this problem by simulating physically accurate fog into clear-weather scenes. We are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset.
  arXiv  Detail & Related papers  (2021-08-11T14:37:54Z)
• Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
  We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process. The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
  arXiv  Detail & Related papers  (2021-04-22T12:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.