Real-World Perturbation Testing of Autonomous Driving Systems

• URL: http://arxiv.org/abs/2607.04953v1
• Date: Mon, 06 Jul 2026 11:34:31 GMT
• Title: Real-World Perturbation Testing of Autonomous Driving Systems
• Abstract summary: We present a large-scale study of 72 camera and LiDAR perturbations, evaluated across three testing modalities.<n>For camera-based systems, perturbations with limited offline impact can still induce unstable control and failures in real-world driving.<n>We show that real-time feasibility is a key constraint in real-world testing.
• Score: 0.9956658791307307
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Autonomous Driving Systems (ADS) must operate reliably under diverse conditions, yet representative data for rare or adverse scenarios is difficult to obtain. Perturbation-based testing is widely used to assess robustness, but most studies focus on offline datasets or simulation, leaving open questions about how such results translate to real-world driving. We present a large-scale study of 72 camera and LiDAR perturbations, evaluated across three testing modalities: offline model-level analysis, hardware-in-the-loop execution, and closed-loop system-level testing on a full-scale autonomous vehicle. The study covers both an end-to-end vision-based driving model and a modular LiDAR-based perception and planning stack. Our results reveal a clear gap between testing levels. For camera-based systems, perturbations with limited offline impact can still induce unstable control and failures in real-world driving. For LiDAR-based systems, degradation is more consistent at the perception level but weakly predictive of system-level failures. Across both modalities, model-level metrics alone are insufficient to identify the most harmful perturbations. We further show that real-time feasibility is a key constraint in real-world testing, and that robustness observations obtained from recorded data do not consistently transfer to closed-loop behavior on a physical vehicle, highlighting the importance of complementary real-world, system-level evaluation.

Related papers

• Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations [122.22391796628408]
  We present Bench2Drive-Robust, to our knowledge the first device-centric robustness benchmark for closed-loop end-to-end autonomous driving.<n>We systematically evaluate deployment-oriented perturbations arising from three major sources: camera-stream failures, ego-state estimation errors, and compute-induced control delay.<n>Our results show that these deployment-related perturbations can substantially degrade closed-loop driving performance.
  arXiv  Detail & Related papers  (2026-05-18T08:45:24Z)
• Unsupervised Learning for Detection of Rare Driving Scenarios [1.5399429731150376]
  This research explores an unsupervised learning framework for detecting rare and extreme driving scenarios.<n>We leverage the recently proposed Deep Isolation Forest (DIF) to identify non-linear and complex anomalies.<n>Our results demonstrate that the proposed approach effectively identifies rare and hazardous driving scenarios.
  arXiv  Detail & Related papers  (2025-12-29T16:35:20Z)
• A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems [0.9956658791307307]
  We compare four representative testing modalities: Software-in-the-Loop (SiL), Vehicle-in-the-Loop (ViL), Mixed-Reality (MR) and full real-world testing.<n>Our results show that while SiL and ViL setups simplify critical aspects of real-world dynamics and sensing, MR testing improves perceptual realism without compromising safety or control.
  arXiv  Detail & Related papers  (2025-09-26T14:08:53Z)
• From Physics to Machine Learning and Back: Part II - Learning and Observational Bias in PHM [52.64097278841485]
  Review examines how incorporating learning and observational biases through physics-informed modeling and data strategies can guide models toward physically consistent and reliable predictions.<n>Fast adaptation methods including meta-learning and few-shot learning are reviewed alongside domain generalization techniques.
  arXiv  Detail & Related papers  (2025-09-25T14:15:43Z)
• Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving [55.96227460521096]
  Vision-Language Models (VLMs) have been integrated into autonomous driving systems to enhance reasoning capabilities.<n>We propose a natural reflection-based backdoor attack targeting VLM systems in autonomous driving scenarios.<n>Our findings uncover a new class of attacks that exploit the stringent real-time requirements of autonomous driving.
  arXiv  Detail & Related papers  (2025-05-09T20:28:17Z)
• Behavioral Cloning Models Reality Check for Autonomous Driving [5.021331908103135]
  This paper presents the real-world validation of state-of-the-art perception systems that utilize Behavior Cloning (BC) for lateral control. The dataset was collected using a scaled research vehicle and tested on various track setups. Experimental results demonstrate that these methods predict steering angles with low error margins in real-time.
  arXiv  Detail & Related papers  (2024-09-11T12:19:38Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• How Do We Fail? Stress Testing Perception in Autonomous Vehicles [40.19326157052966]
  This paper presents a method for characterizing failures of LiDAR-based perception systems for autonomous vehicles in adverse weather conditions. We develop a methodology based in reinforcement learning to find likely failures in object tracking and trajectory prediction due to sequences of disturbances.
  arXiv  Detail & Related papers  (2022-03-26T20:48:09Z)
• Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving [100.57791628642624]
  We introduce a safety guaranteed learning framework for vision-based end-to-end autonomous driving. We design a learning system equipped with differentiable control barrier functions (dCBFs) that is trained end-to-end by gradient descent.
  arXiv  Detail & Related papers  (2022-03-04T16:14:33Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.