A Roadmap for Simulation-Based Testing of Autonomous Cyber-Physical Systems: Challenges and Future Direction

• URL: http://arxiv.org/abs/2405.01064v1
• Date: Thu, 2 May 2024 07:42:33 GMT
• Title: A Roadmap for Simulation-Based Testing of Autonomous Cyber-Physical Systems: Challenges and Future Direction
• Authors: Christian Birchler, Sajad Khatiri, Pooja Rani, Timo Kehrer, Sebastiano Panichella,
• Abstract summary: This paper pioneers a strategic roadmap for simulation-based testing of autonomous systems. Our paper discusses the relevant challenges and obstacles of ACPSs, focusing on test automation and quality assurance.
• Score: 5.742965094549775
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As the era of autonomous cyber-physical systems (ACPSs), such as unmanned aerial vehicles and self-driving cars, unfolds, the demand for robust testing methodologies is key to realizing the adoption of such systems in real-world scenarios. However, traditional software testing paradigms face unprecedented challenges in ensuring the safety and reliability of these systems. In response, this paper pioneers a strategic roadmap for simulation-based testing of ACPSs, specifically focusing on autonomous systems. Our paper discusses the relevant challenges and obstacles of ACPSs, focusing on test automation and quality assurance, hence advocating for tailored solutions to address the unique demands of autonomous systems. While providing concrete definitions of test cases within simulation environments, we also accentuate the need to create new benchmark assets and the development of automated tools tailored explicitly for autonomous systems in the software engineering community. This paper not only highlights the relevant, pressing issues the software engineering community should focus on (in terms of practices, expected automation, and paradigms), but it also outlines ways to tackle them. By outlining the various domains and challenges of simulation-based testing/development for ACPSs, we provide directions for future research efforts.

Related papers

• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• Runtime Verification and Field-based Testing for ROS-based Robotic Systems [8.675312581079039]
  No clear guidance exists for architecting ROS-based systems to enable runtime verification and field-based testing. This paper aims to fill this gap by providing guidelines to help developers and quality assurance (QA) teams develop, verify, or test their robots in the field.
  arXiv  Detail & Related papers  (2024-04-17T15:52:29Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• When Authentication Is Not Enough: On the Security of Behavioral-Based Driver Authentication Systems [53.2306792009435]
  We develop two lightweight driver authentication systems based on Random Forest and Recurrent Neural Network architectures. We are the first to propose attacks against these systems by developing two novel evasion attacks, SMARTCAN and GANCAN. Through our contributions, we aid practitioners in safely adopting these systems, help reduce car thefts, and enhance driver security.
  arXiv  Detail & Related papers  (2023-06-09T14:33:26Z)
• Rethinking Certification for Higher Trust and Ethical Safeguarding of Autonomous Systems [6.24907186790431]
  We discuss the motivation for the need to modify the current certification processes for autonomous driving systems. We identify a number of issues with the proposed certification strategies, which may impact the systems substantially.
  arXiv  Detail & Related papers  (2023-03-16T15:19:25Z)
• AmbieGen: A Search-based Framework for Autonomous Systems Testing [12.93632948681342]
  AmbieGen is a search-based test case generation framework for autonomous systems. It supports test case generation for autonomous robots and autonomous car lane keeping assist systems.
  arXiv  Detail & Related papers  (2023-01-01T23:42:32Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• AI-as-a-Service Toolkit for Human-Centered Intelligence in Autonomous Driving [13.575818872875637]
  This paper presents a proof-of-concept implementation of the AI-as-a-service toolkit developed within the H2020 TEACHING project. It implements an autonomous driving personalization system according to the output of an automatic driver's stress recognition algorithm.
  arXiv  Detail & Related papers  (2022-02-03T15:41:43Z)
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System [78.60415450507706]
  Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty.
  arXiv  Detail & Related papers  (2021-07-28T10:28:05Z)
• Data Driven Testing of Cyber Physical Systems [12.93632948681342]
  We propose an approach to automatically generate fault-revealing test cases for CPS. Data collected from an application managing a smart building have been used to learn models of the environment.
  arXiv  Detail & Related papers  (2021-02-23T04:55:10Z)
• A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
  This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
  arXiv  Detail & Related papers  (2020-11-17T22:24:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.