Representation Improvement in Latent Space for Search-Based Testing of Autonomous Robotic Systems

• URL: http://arxiv.org/abs/2503.20642v1
• Date: Wed, 26 Mar 2025 15:34:42 GMT
• Title: Representation Improvement in Latent Space for Search-Based Testing of Autonomous Robotic Systems
• Authors: Dmytro Humeniuk, Foutse Khomh,
• Abstract summary: Testing autonomous robotic systems, such as self-driving cars and unmanned aerial vehicles, is challenging due to their interaction with unpredictable environments.<n>A number of search-based approaches were proposed to generate test scenarios more efficiently.<n>We propose RILaST, which enhances test representation by mapping it to the latent space of a variational autoencoder.
• Score: 13.431632585434384
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Testing autonomous robotic systems, such as self-driving cars and unmanned aerial vehicles, is challenging due to their interaction with highly unpredictable environments. A common practice is to first conduct simulation-based testing, which, despite reducing real-world risks, remains time-consuming and resource-intensive due to the vast space of possible test scenarios. A number of search-based approaches were proposed to generate test scenarios more efficiently. A key aspect of any search-based test generation approach is the choice of representation used during the search process. However, existing methods for improving test scenario representation remain limited. We propose RILaST (Representation Improvement in Latent Space for Search-Based Testing) approach, which enhances test representation by mapping it to the latent space of a variational autoencoder. We evaluate RILaST on two use cases, including autonomous drone and autonomous lane-keeping assist system. The obtained results show that RILaST allows finding between 3 to 4.6 times more failures than baseline approaches, achieving a high level of test diversity.

Related papers

• ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning [78.42927884000673]
  ExACT is an approach to combine test-time search and self-learning to build o1-like models for agentic applications.<n>We first introduce Reflective Monte Carlo Tree Search (R-MCTS), a novel test time algorithm designed to enhance AI agents' ability to explore decision space on the fly.<n>Next, we introduce Exploratory Learning, a novel learning strategy to teach agents to search at inference time without relying on any external search algorithms.
  arXiv  Detail & Related papers  (2024-10-02T21:42:35Z)
• Diversity-guided Search Exploration for Self-driving Cars Test Generation through Frenet Space Encoding [4.135985106933988]
  The rise of self-driving cars (SDCs) presents important safety challenges to address in dynamic environments. While field testing is essential, current methods lack diversity in assessing critical SDC scenarios. We show that the likelihood of leading to an out-of-bound condition can be learned by the deep-learning vanilla transformer model.
  arXiv  Detail & Related papers  (2024-01-26T06:57:00Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events [1.84926694477846]
  We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles. Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
  arXiv  Detail & Related papers  (2023-08-28T13:09:00Z)
• Reinforcement learning informed evolutionary search for autonomous systems testing [15.210312666486029]
  We propose augmenting the evolutionary search (ES) with a reinforcement learning (RL) agent trained using surrogate rewards derived from domain knowledge. In our approach, known as RIGAA, we first train an RL agent to learn useful constraints of the problem and then use it to produce a certain part of the initial population of the search algorithm. We evaluate RIGAA on two case studies: maze generation for an autonomous ant robot and road topology generation for an autonomous vehicle lane keeping assist system.
  arXiv  Detail & Related papers  (2023-08-24T13:11:07Z)
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition [65.84978547406753]
  Test-time Adaptation aims to adapt the model trained on source domains to yield better predictions for test samples. Single-Utterance Test-time Adaptation (SUTA) is the first TTA study in speech area to our best knowledge.
  arXiv  Detail & Related papers  (2022-03-27T06:38:39Z)
• Machine Learning Testing in an ADAS Case Study Using Simulation-Integrated Bio-Inspired Search-Based Testing [7.5828169434922]
  Deeper generates failure-revealing test scenarios for testing a deep neural network-based lane-keeping system. In the newly proposed version, we utilize a new set of bio-inspired search algorithms, genetic algorithm (GA), $(mu+lambda)$ and $(mu,lambda)$ evolution strategies (ES), and particle swarm optimization (PSO) Our evaluation shows the newly proposed test generators in Deeper represent a considerable improvement on the previous version.
  arXiv  Detail & Related papers  (2022-03-22T20:27:40Z)
• A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation [26.10081199009559]
  Testing the system on the road is the closest to real-world and desirable approach, but it is incredibly costly. A popular alternative is to evaluate an ADS's performance in some well-designed challenging scenarios, a.k.a. scenario-based testing. High-fidelity simulators have been widely used in this setting to maximize flexibility and convenience in testing what-if scenarios.
  arXiv  Detail & Related papers  (2021-12-02T03:41:33Z)
• Efficient and Effective Generation of Test Cases for Pedestrian Detection -- Search-based Software Testing of Baidu Apollo in SVL [14.482670650074885]
  This paper presents a study on testing pedestrian detection and emergency braking system of the Baidu Apollo autonomous driving platform within the SVL simulator. We propose an evolutionary automated test generation technique that generates failure-revealing scenarios for Apollo in the SVL environment. In order to demonstrate the efficiency and effectiveness of our approach, we also report the results from a baseline random generation technique.
  arXiv  Detail & Related papers  (2021-09-16T13:11:53Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction [88.0416857308144]
  We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. We directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing.
  arXiv  Detail & Related papers  (2020-08-13T17:20:02Z)
• AutoOD: Automated Outlier Detection via Curiosity-guided Search and Self-imitation Learning [72.99415402575886]
  Outlier detection is an important data mining task with numerous practical applications. We propose AutoOD, an automated outlier detection framework, which aims to search for an optimal neural network model. Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoOD achieves the best performance.
  arXiv  Detail & Related papers  (2020-06-19T18:57:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.