A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap of Radar Perception for Autonomous Driving

• URL: http://arxiv.org/abs/2106.08372v1
• Date: Tue, 15 Jun 2021 18:51:39 GMT
• Title: A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap of Radar Perception for Autonomous Driving
• Authors: Anthony Ngo, Max Paul Bauer and Michael Resch
• Abstract summary: In order to rely on virtual tests the employed sensor models have to be validated. There exists no sound method to measure this simulation-to-reality gap of radar perception. We have shown the effectiveness of the proposed approach in terms of providing an in-depth sensor model assessment.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the increasing safety validation requirements for the release of a self-driving car, alternative approaches, such as simulation-based testing, are emerging in addition to conventional real-world testing. In order to rely on virtual tests the employed sensor models have to be validated. For this reason, it is necessary to quantify the discrepancy between simulation and reality in order to determine whether a certain fidelity is sufficient for a desired intended use. There exists no sound method to measure this simulation-to-reality gap of radar perception for autonomous driving. We address this problem by introducing a multi-layered evaluation approach, which consists of a combination of an explicit and an implicit sensor model evaluation. The former directly evaluates the realism of the synthetically generated sensor data, while the latter refers to an evaluation of a downstream target application. In order to demonstrate the method, we evaluated the fidelity of three typical radar model types (ideal, data-driven, ray tracing-based) and their applicability for virtually testing radar-based multi-object tracking. We have shown the effectiveness of the proposed approach in terms of providing an in-depth sensor model assessment that renders existing disparities visible and enables a realistic estimation of the overall model fidelity across different scenarios.

Related papers

• XLD: A Cross-Lane Dataset for Benchmarking Novel Driving View Synthesis [84.23233209017192]
  This paper presents a novel driving view synthesis dataset and benchmark specifically designed for autonomous driving simulations. The dataset is unique as it includes testing images captured by deviating from the training trajectory by 1-4 meters. We establish the first realistic benchmark for evaluating existing NVS approaches under front-only and multi-camera settings.
  arXiv  Detail & Related papers  (2024-06-26T14:00:21Z)
• Can you see me now? Blind spot estimation for autonomous vehicles using scenario-based simulation with random reference sensors [5.910402196056647]
  A Monte Carlo-based reference sensor simulation enables us to accurately estimate blind spot size as a metric of coverage. Our method leverages point clouds from LiDAR sensors or camera depth images from high-fidelity simulations of target scenarios to provide accurate and actionable visibility estimates.
  arXiv  Detail & Related papers  (2024-02-01T10:14:53Z)
• A Diffusion-Model of Joint Interactive Navigation [14.689298253430568]
  We present DJINN - a diffusion based method of generating traffic scenarios. Our approach jointly diffuses the trajectories of all agents, conditioned on a flexible set of state observations from the past, present, or future. We show how DJINN flexibly enables direct test-time sampling from a variety of valuable conditional distributions.
  arXiv  Detail & Related papers  (2023-09-21T22:10:20Z)
• Reinforcement Learning with Human Feedback for Realistic Traffic Simulation [53.85002640149283]
  Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge. This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
  arXiv  Detail & Related papers  (2023-09-01T19:29:53Z)
• Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehicles [45.27200446670184]
  We propose a perception imitation method to simulate results of a certain perception model, and discuss a new route of autonomous driving simulator without data synthesis. Experiments show that our method is effective to model the behavior of learning-based perception model, and can be further applied in the proposed simulation route smoothly.
  arXiv  Detail & Related papers  (2023-04-19T01:27:02Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• Deep Evaluation Metric: Learning to Evaluate Simulated Radar Point Clouds for Virtual Testing of Autonomous Driving [0.0]
  The usage of environment sensor models for virtual testing is a promising approach to reduce the testing effort of autonomous driving. In this work, we train a neural network to distinguish real and simulated radar sensor data. We propose the classifier's confidence score for the real radar point cloud' class as a metric to determine the degree of fidelity of synthetically generated radar data.
  arXiv  Detail & Related papers  (2021-04-14T11:04:50Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction [88.0416857308144]
  We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. We directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing.
  arXiv  Detail & Related papers  (2020-08-13T17:20:02Z)
• A Sensitivity Analysis Approach for Evaluating a Radar Simulation for Virtual Testing of Autonomous Driving Functions [0.0]
  We introduce a sensitivity analysis approach for developing and evaluating a radar simulation. A modular radar system simulation is presented and parameterized to conduct a sensitivity analysis. We compare the output from the radar model to real driving measurements to ensure a realistic model behavior.
  arXiv  Detail & Related papers  (2020-08-06T15:51:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.