Testing the Fault-Tolerance of Multi-Sensor Fusion Perception in Autonomous Driving Systems

• URL: http://arxiv.org/abs/2504.13420v1
• Date: Fri, 18 Apr 2025 02:37:55 GMT
• Title: Testing the Fault-Tolerance of Multi-Sensor Fusion Perception in Autonomous Driving Systems
• Authors: Haoxiang Tian, Wenqiang Ding, Xingshuo Han, Guoquan Wu, An Guo, Junqi Zhang. Wei Chen, Jun Wei, Tianwei Zhang,
• Abstract summary: We build fault models for cameras and LiDAR in AVs and inject them into the MSF perception-based ADS to test its behaviors in test scenarios.<n>We design a feedback-guided differential fuzzer to discover the safety violations of MSF perception-based ADS caused by the injected sensor faults.
• Score: 14.871090150807929
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-level Autonomous Driving Systems (ADSs), such as Google Waymo and Baidu Apollo, typically rely on multi-sensor fusion (MSF) based approaches to perceive their surroundings. This strategy increases perception robustness by combining the respective strengths of the camera and LiDAR and directly affects the safety-critical driving decisions of autonomous vehicles (AVs). However, in real-world autonomous driving scenarios, cameras and LiDAR are subject to various faults, which can probably significantly impact the decision-making and behaviors of ADSs. Existing MSF testing approaches only discovered corner cases that the MSF-based perception cannot accurately detected by MSF-based perception, while lacking research on how sensor faults affect the system-level behaviors of ADSs. To address this gap, we conduct the first exploration of the fault tolerance of MSF perception-based ADS for sensor faults. In this paper, we systematically and comprehensively build fault models for cameras and LiDAR in AVs and inject them into the MSF perception-based ADS to test its behaviors in test scenarios. To effectively and efficiently explore the parameter spaces of sensor fault models, we design a feedback-guided differential fuzzer to discover the safety violations of MSF perception-based ADS caused by the injected sensor faults. We evaluate FADE on the representative and practical industrial ADS, Baidu Apollo. Our evaluation results demonstrate the effectiveness and efficiency of FADE, and we conclude some useful findings from the experimental results. To validate the findings in the physical world, we use a real Baidu Apollo 6.0 EDU autonomous vehicle to conduct the physical experiments, and the results show the practical significance of our findings.

Related papers

• Knowledge Graphs as World Models for Semantic Material-Aware Obstacle Handling in Autonomous Vehicles [0.03946288852327085]
  This study suggests combining sensors with a knowledge graph (KG)-based world model to improve AVs' comprehension of physical material qualities. Using the CARLA autonomous driving simulator, we evaluated AV performance with and without KG integration. The KG-based method improves obstacle management, which allows AVs to use material qualities to make better decisions about when to change lanes or apply emergency braking.
  arXiv  Detail & Related papers  (2025-03-27T07:46:45Z)
• Hi-ALPS -- An Experimental Robustness Quantification of Six LiDAR-based Object Detection Systems for Autonomous Driving [49.64902130083662]
  3D object detection systems (OD) play a key role in the driving decisions of autonomous vehicles. Adversarial examples are small, sometimes sophisticated perturbations in the input data that change, i.e. falsify, the prediction of the OD. We quantify the robustness of six state-of-the-art 3D OD under different types of perturbations.
  arXiv  Detail & Related papers  (2025-03-21T14:17:02Z)
• Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Models [7.649716717097431]
  We explore the integration of generative artificial intelligence techniques with physics-based simulators to enhance Autonomous Driving Systems (ADS) system-level testing.<n>Our study evaluates the effectiveness and computational overhead of three generative strategies based on diffusion models.<n>We employ a novel automated detector for invalid inputs based on semantic segmentation to ensure semantic preservation and realism of the neural generated images.
  arXiv  Detail & Related papers  (2024-09-20T17:09:45Z)
• Evaluating the Robustness of LiDAR-based 3D Obstacles Detection and Its Impacts on Autonomous Driving Systems [4.530172587010801]
  We study the impact of built-in inaccuracies in LiDAR sensors on LiDAR-3D obstacle detection models. We apply ET to evaluate the robustness of five classic LiDAR-3D obstacle detection models. We find that even very subtle changes in point cloud data may introduce a non-trivial decrease in the detection performance.
  arXiv  Detail & Related papers  (2024-08-24T19:10:07Z)
• LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusions [61.87108000328186]
  Lane detection (LD) is an essential component of autonomous driving systems, providing fundamental functionalities like adaptive cruise control and automated lane centering. Existing LD benchmarks primarily focus on evaluating common cases, neglecting the robustness of LD models against environmental illusions. This paper studies the potential threats caused by these environmental illusions to LD and establishes the first comprehensive benchmark LanEvil.
  arXiv  Detail & Related papers  (2024-06-03T02:12:27Z)
• Experimental Evaluation of Road-Crossing Decisions by Autonomous Wheelchairs against Environmental Factors [42.90509901417468]
  We focus on the fine-tuning of tracking performance and on its experimental evaluation against outdoor environmental factors. We show that the approach can be adopted to evaluate video tracking and event detection robustness against outdoor environmental factors.
  arXiv  Detail & Related papers  (2024-05-27T08:43:26Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• Efficient Real-time Smoke Filtration with 3D LiDAR for Search and Rescue with Autonomous Heterogeneous Robotic Systems [56.838297900091426]
  Smoke and dust affect the performance of any mobile robotic platform due to their reliance on onboard perception systems. This paper proposes a novel modular computation filtration pipeline based on intensity and spatial information.
  arXiv  Detail & Related papers  (2023-08-14T16:48:57Z)
• Detecting Safety Problems of Multi-Sensor Fusion in Autonomous Driving [18.39664775350204]
  Multi-sensor fusion (MSF) is used to fuse the sensor inputs and produce a more reliable understanding of the surroundings. MSF methods in an industry-grade Advanced Driver-Assistance System (ADAS) can mislead the car control and result in serious safety hazards. We develop a novel evolutionary-based domain-specific search framework, FusionFuzz, for the efficient detection of fusion errors.
  arXiv  Detail & Related papers  (2021-09-14T02:35:34Z)
• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks [62.923992740383966]
  We present the first study of security issues of MSF-based perception in AD systems. We generate a physically-realizable, adversarial 3D-printed object that misleads an AD system to fail in detecting it and thus crash into it. Our results show that the attack achieves over 90% success rate across different object types and MSF.
  arXiv  Detail & Related papers  (2021-06-17T05:11:07Z)
• Towards robust sensing for Autonomous Vehicles: An adversarial perspective [82.83630604517249]
  It is of primary importance that the resulting decisions are robust to perturbations. Adversarial perturbations are purposefully crafted alterations of the environment or of the sensory measurements. A careful evaluation of the vulnerabilities of their sensing system(s) is necessary in order to build and deploy safer systems.
  arXiv  Detail & Related papers  (2020-07-14T05:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.