Automated System-level Testing of Unmanned Aerial Systems

• URL: http://arxiv.org/abs/2403.15857v2
• Date: Fri, 2 Aug 2024 11:36:14 GMT
• Title: Automated System-level Testing of Unmanned Aerial Systems
• Authors: Hassan Sartaj, Asmar Muqeet, Muhammad Zohaib Iqbal, Muhammad Uzair Khan,
• Abstract summary: A major requirement of international safety standards is to perform rigorous system-level testing of avionics software systems. The proposed approach (AITester) utilizes model-based testing and artificial intelligence (AI) techniques to automatically generate, execute, and evaluate various test scenarios.
• Score: 2.2249176072603634
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unmanned aerial systems (UAS) rely on various avionics systems that are safety-critical and mission-critical. A major requirement of international safety standards is to perform rigorous system-level testing of avionics software systems. The current industrial practice is to manually create test scenarios, manually/automatically execute these scenarios using simulators, and manually evaluate outcomes. The test scenarios typically consist of setting certain flight or environment conditions and testing the system under test in these settings. The state-of-the-art approaches for this purpose also require manual test scenario development and evaluation. In this paper, we propose a novel approach to automate the system-level testing of the UAS. The proposed approach (AITester) utilizes model-based testing and artificial intelligence (AI) techniques to automatically generate, execute, and evaluate various test scenarios. The test scenarios are generated on the fly, i.e., during test execution based on the environmental context at runtime. The approach is supported by a toolset. We empirically evaluate the proposed approach on two core components of UAS, an autopilot system of an unmanned aerial vehicle (UAV) and cockpit display systems (CDS) of the ground control station (GCS). The results show that the AITester effectively generates test scenarios causing deviations from the expected behavior of the UAV autopilot and reveals potential flaws in the GCS-CDS.

Related papers

• Automated Soap Opera Testing Directed by LLMs and Scenario Knowledge: Feasibility, Challenges, and Road Ahead [43.15092098658384]
  Exploratory testing (ET) harnesses tester's knowledge, creativity, and experience to create varying tests that uncover unexpected bugs from the end-user's perspective. We explore the feasibility, challenges and road ahead of automated scenario-based ET (a.k.a soap opera testing)
  arXiv  Detail & Related papers  (2024-12-11T17:57:23Z)
• SAVME: Efficient Safety Validation for Autonomous Systems Using Meta-Learning [36.896695278624776]
  We propose a Bayesian approach that integrates meta-learning strategies with a multi-armed bandit framework. We showcase our methodology using a cutting-edge 3D driving simulator, incorporating 16 fidelity settings for an autonomous vehicle stack. Our approach achieves a significant speedup, up to 18 times faster compared to traditional methods.
  arXiv  Detail & Related papers  (2023-09-21T20:41:47Z)
• A Requirements-Driven Platform for Validating Field Operations of Small Uncrewed Aerial Vehicles [48.67061953896227]
  DroneReqValidator (DRV) allows sUAS developers to define the operating context, configure multi-sUAS mission requirements, specify safety properties, and deploy their own custom sUAS applications in a high-fidelity 3D environment. The DRV Monitoring system collects runtime data from sUAS and the environment, analyzes compliance with safety properties, and captures violations.
  arXiv  Detail & Related papers  (2023-07-01T02:03:49Z)
• AmbieGen: A Search-based Framework for Autonomous Systems Testing [12.93632948681342]
  AmbieGen is a search-based test case generation framework for autonomous systems. It supports test case generation for autonomous robots and autonomous car lane keeping assist systems.
  arXiv  Detail & Related papers  (2023-01-01T23:42:32Z)
• A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation [26.10081199009559]
  Testing the system on the road is the closest to real-world and desirable approach, but it is incredibly costly. A popular alternative is to evaluate an ADS's performance in some well-designed challenging scenarios, a.k.a. scenario-based testing. High-fidelity simulators have been widely used in this setting to maximize flexibility and convenience in testing what-if scenarios.
  arXiv  Detail & Related papers  (2021-12-02T03:41:33Z)
• Complete Agent-driven Model-based System Testing for Autonomous Systems [0.0]
  A novel approach to testing complex autonomous transportation systems is described. It is intended to mitigate some of the most critical problems regarding verification and validation.
  arXiv  Detail & Related papers  (2021-10-25T01:55:24Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction [88.0416857308144]
  We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. We directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing.
  arXiv  Detail & Related papers  (2020-08-13T17:20:02Z)
• Towards Automated Safety Coverage and Testing for Autonomous Vehicles with Reinforcement Learning [0.3683202928838613]
  Validation puts the autonomous vehicle system to the test in scenarios or situations that the system would likely encounter in everyday driving. We propose using reinforcement learning (RL) to generate failure examples and unexpected traffic situations for the AV software implementation.
  arXiv  Detail & Related papers  (2020-05-22T19:00:38Z)
• Search-based Test-Case Generation by Monitoring Responsibility Safety Rules [2.1270496914042996]
  We propose a method for screening and classifying simulation-based driving test data to be used for training and testing controllers. Our framework is distributed with the publicly available S-TALIRO and Sim-ATAV tools.
  arXiv  Detail & Related papers  (2020-04-25T10:10:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.