Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directions

• URL: http://arxiv.org/abs/2512.11887v1
• Date: Tue, 09 Dec 2025 06:33:27 GMT
• Title: Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directions
• Authors: Yihan Liao, Jingyu Zhang, Jacky Keung, Yan Xiao, Yurou Dai,
• Abstract summary: Autonomous driving systems (ADSs) promise improved transportation efficiency and safety, yet ensuring their reliability in complex real-world environments remains a critical challenge.<n>This study investigates current ADS testing practices for both modular and end-to-end systems.<n>We review major testing techniques and consider emerging factors such as Vehicle-to-Everything (V2X) communication and foundation models.
• Score: 16.701909787572724
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous driving systems (ADSs) promise improved transportation efficiency and safety, yet ensuring their reliability in complex real-world environments remains a critical challenge. Effective testing is essential to validate ADS performance and reduce deployment risks. This study investigates current ADS testing practices for both modular and end-to-end systems, identifies key demands from industry practitioners and academic researchers, and analyzes the gaps between existing research and real-world requirements. We review major testing techniques and further consider emerging factors such as Vehicle-to-Everything (V2X) communication and foundation models, including large language models and vision foundation models, to understand their roles in enhancing ADS testing. We conducted a large-scale survey with 100 participants from both industry and academia. Survey questions were refined through expert discussions, followed by quantitative and qualitative analyses to reveal key trends, challenges, and unmet needs. Our results show that existing ADS testing techniques struggle to comprehensively evaluate real-world performance, particularly regarding corner case diversity, the simulation to reality gap, the lack of systematic testing criteria, exposure to potential attacks, practical challenges in V2X deployment, and the high computational cost of foundation model-based testing. By further analyzing participant responses together with 105 representative studies, we summarize the current research landscape and highlight major limitations. This study consolidates critical research gaps in ADS testing and outlines key future research directions, including comprehensive testing criteria, cross-model collaboration in V2X systems, cross-modality adaptation for foundation model-based testing, and scalable validation frameworks for large-scale ADS evaluation.

Related papers

• Can AI Generate more Comprehensive Test Scenarios? Review on Automated Driving Systems Test Scenario Generation Methods [19.39586739934126]
  This review systematically analyzes 31 primary studies,and 10 surveys identified through a comprehensive search spanning 20152025.<n>Traditional approaches rely on expert knowledge,ontologies,and naturalistic driving or accident data,while recent developments leverage generative models,including large language models, adversarial networks,diffusion models,and reinforcement learning frameworks,to synthesize diverse and safety-critical scenarios.
  arXiv  Detail & Related papers  (2025-12-17T13:14:15Z)
• Let the Barbarians In: How AI Can Accelerate Systems Performance Research [80.43506848683633]
  We term this iterative cycle of generation, evaluation, and refinement AI-Driven Research for Systems.<n>We demonstrate that ADRS-generated solutions can match or even outperform human state-of-the-art designs.
  arXiv  Detail & Related papers  (2025-12-16T18:51:23Z)
• ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning [118.46980291324148]
  ATLAS is a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems.<n>Its key features include: High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage.<n>Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities.
  arXiv  Detail & Related papers  (2025-11-18T11:13:06Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• Generative AI for Testing of Autonomous Driving Systems: A Survey [13.226510198306885]
  Autonomous driving systems (ADS) have been an active area of research, with the potential to deliver significant benefits to society.<n>Different testing approaches are required, and achieving effective and efficient testing of ADS remains an open challenge.<n>generative AI has emerged as a powerful tool across many domains, and it is increasingly being applied to ADS testing due to its ability to interpret context.<n>This survey provides an overview and practical insights into the use of generative AI for testing ADS, highlights existing challenges, and outlines directions for future research in this rapidly evolving field.
  arXiv  Detail & Related papers  (2025-08-27T13:40:14Z)
• AI-Driven Tools in Modern Software Quality Assurance: An Assessment of Benefits, Challenges, and Future Directions [0.0]
  The research aims to assess the benefits, challenges, and prospects of integrating modern AI-oriented tools into quality assurance processes.<n>The research demonstrates AI's transformative potential for QA but highlights the importance of a strategic approach to implementing these technologies.
  arXiv  Detail & Related papers  (2025-06-19T20:22:47Z)
• Anomaly Detection and Generation with Diffusion Models: A Survey [51.61574868316922]
  Anomaly detection (AD) plays a pivotal role across diverse domains, including cybersecurity, finance, healthcare, and industrial manufacturing.<n>Recent advancements in deep learning, specifically diffusion models (DMs), have sparked significant interest.<n>This survey aims to guide researchers and practitioners in leveraging DMs for innovative AD solutions across diverse applications.
  arXiv  Detail & Related papers  (2025-06-11T03:29:18Z)
• A Survey on the Application of Large Language Models in Scenario-Based Testing of Automated Driving Systems [6.608557716494977]
  The paper concludes by outlining five open challenges and potential research directions.<n>The emergence of Large Language Models (LLMs) has introduced new opportunities to reinforce scenario-based testing.
  arXiv  Detail & Related papers  (2025-05-22T12:25:44Z)
• SoK: On Closing the Applicability Gap in Automated Vulnerability Detection [0.18846515534317265]
  Automated Vulnerability Detection (AVD) aims to autonomously analyze source code to identify vulnerabilities.<n>This paper addresses two primary research questions: How is current AVD research distributed across its core components, and what key areas should future research target to bridge the gap in the practical applicability of AVD throughout software development?<n>We conduct a systematization over 79 AVD articles and 17 empirical studies, analyzing them across five core components: task formulation and granularity, input programming languages and representations, detection approaches and key solutions, evaluation metrics and datasets, and reported performance.
  arXiv  Detail & Related papers  (2024-12-15T14:01:41Z)
• AI-Compass: A Comprehensive and Effective Multi-module Testing Tool for AI Systems [26.605694684145313]
  In this study, we design and implement a testing tool, tool, to comprehensively and effectively evaluate AI systems. The tool extensively assesses adversarial robustness, model interpretability, and performs neuron analysis. Our research sheds light on a general solution for AI systems testing landscape.
  arXiv  Detail & Related papers  (2024-11-09T11:15:17Z)
• Failure Diagnosis in Microservice Systems: A Comprehensive Survey and Analysis [10.92325792850306]
  This survey provides an exhaustive review of 98 scientific papers from 2003 to the present.<n>It includes a thorough examination and elucidation of the fundamental concepts, system architecture, and problem statement.<n>It also includes a qualitative analysis of the dimensions, providing an in-depth discussion of current best practices and future directions.
  arXiv  Detail & Related papers  (2024-06-27T10:25:37Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Quality meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing [60.38182654847399]
  Computerized Adaptive Testing (CAT) is emerging as a promising testing application in many scenarios. We propose a novel framework, namely Model-Agnostic Adaptive Testing (MAAT) for CAT solution.
  arXiv  Detail & Related papers  (2021-01-15T06:48:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.