OpenSBT: A Modular Framework for Search-based Testing of Automated Driving Systems

• URL: http://arxiv.org/abs/2306.10296v2
• Date: Thu, 2 Nov 2023 10:15:23 GMT
• Title: OpenSBT: A Modular Framework for Search-based Testing of Automated Driving Systems
• Authors: Lev Sorokin, Tiziano Munaro, Damir Safin, Brian Hsuan-Cheng Liao, Adam Molin
• Abstract summary: Search-based software testing (SBT) is an effective and efficient approach for testing automated driving systems (ADS) We present OpenSBT, an open-source, modular and framework to facilitate the SBT of ADS.
• Score: 0.4499833362998489
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Search-based software testing (SBT) is an effective and efficient approach for testing automated driving systems (ADS). However, testing pipelines for ADS testing are particularly challenging as they involve integrating complex driving simulation platforms and establishing communication protocols and APIs with the desired search algorithm. This complexity prevents a wide adoption of SBT and thorough empirical comparative experiments with different simulators and search approaches. We present OpenSBT, an open-source, modular and extensible framework to facilitate the SBT of ADS. With OpenSBT, it is possible to integrate simulators with an embedded system under test, search algorithms and fitness functions for testing. We describe the architecture and show the usage of our framework by applying different search algorithms for testing Automated Emergency Braking Systems in CARLA as well in the high-fidelity Prescan simulator in collaboration with our industrial partner DENSO. OpenSBT is available at https://git.fortiss.org/opensbt. A demo video is provided here: https://youtu.be/6csl\_UAOD\_4.

Related papers

• OpenTAD: A Unified Framework and Comprehensive Study of Temporal Action Detection [86.30994231610651]
  Temporal action detection (TAD) is a fundamental video understanding task that aims to identify human actions and localize their temporal boundaries in videos. We propose textbfOpenTAD, a unified TAD framework consolidating 16 different TAD methods and 9 standard datasets into a modular framework. Minimal effort is required to replace one module with a different design, train a feature-based TAD model in end-to-end mode, or switch between the two.
  arXiv  Detail & Related papers  (2025-02-27T18:32:27Z)
• Automating a Complete Software Test Process Using LLMs: An Automotive Case Study [10.245216059506236]
  Vehicle API testing verifies whether the interactions between a vehicle's internal systems and external applications meet expectations. This paper presents a system designed for the automated testing of in-vehicle APIs.
  arXiv  Detail & Related papers  (2025-02-06T12:10:01Z)
• AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL [46.65963514391019]
  AutoRestTest is a novel tool that integrates the Semantic Property Dependency Graph (SPDG) with Multi-Agent Reinforcement Learning (MARL) and large language models (LLMs) for effective REST API testing.
  arXiv  Detail & Related papers  (2025-01-15T05:54:33Z)
• SocialED: A Python Library for Social Event Detection [53.928241775629566]
  SocialED is a comprehensive, open-source Python library designed to support social event detection (SED) tasks. It provides a unified API with detailed documentation, offering researchers and practitioners a complete solution for event detection in social media. SocialED supports a wide range of preprocessing techniques, such as graph construction and tokenization, and includes standardized interfaces for training models and making predictions.
  arXiv  Detail & Related papers  (2024-12-18T03:37:47Z)
• DriveTester: A Unified Platform for Simulation-Based Autonomous Driving Testing [24.222344794923558]
  DriveTester is a unified simulation-based testing platform built on Apollo. It provides a consistent and reliable environment, integrates a lightweight traffic simulator, and incorporates various state-of-the-art ADS testing techniques.
  arXiv  Detail & Related papers  (2024-12-17T08:24:05Z)
• Automated Test-Case Generation for REST APIs Using Model Inference Search Heuristic [15.625240669567479]
  EvoMaster is a tool that uses Evolutionary Algorithms (EAs) to automatically generate test cases for REST APIs. We propose a new search (MISH) that uses real-time automaton learning to guide the automated test case generation process. MISH learns a representation of the systemwide behavior, allowing to define the fitness of a test case based on the path it traverses within the inferred.
  arXiv  Detail & Related papers  (2024-12-04T16:00:14Z)
• Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning [50.26965628047682]
  Adapting pre-trained models to open classes is a challenging problem in machine learning. In this paper, we consider combining the advantages of both and come up with a test-time prompt tuning approach. Our proposed method outperforms all comparison methods on average considering both base and new classes.
  arXiv  Detail & Related papers  (2024-08-29T12:34:01Z)
• KAT: Dependency-aware Automated API Testing with Large Language Models [1.7264233311359707]
  KAT (Katalon API Testing) is a novel AI-driven approach that autonomously generates test cases to validate APIs. Our evaluation of KAT using 12 real-world services shows that it can improve validation coverage, detect more undocumented status codes, and reduce false positives in these services.
  arXiv  Detail & Related papers  (2024-07-14T14:48:18Z)
• The Power of Resets in Online Reinforcement Learning [73.64852266145387]
  We explore the power of simulators through online reinforcement learning with local simulator access (or, local planning) We show that MDPs with low coverability can be learned in a sample-efficient fashion with only $Qstar$-realizability. We show that the notorious Exogenous Block MDP problem is tractable under local simulator access.
  arXiv  Detail & Related papers  (2024-04-23T18:09:53Z)
• Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models [114.69732301904419]
  We present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text. Our approach demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations.
  arXiv  Detail & Related papers  (2023-10-26T17:56:35Z)
• TEASER: Simulation-based CAN Bus Regression Testing for Self-driving Cars Software [5.77326389907799]
  TEASER is a tool that generates realistic CAN signals for SDCs obtained from sensors from state-of-the-art car simulators. We integrated TEASER in a Continous Integration (CI) pipeline configured with Jenkins. Our evaluation shows the ability of TEASER to generate and execute CI test cases that expose simulation-based faults.
  arXiv  Detail & Related papers  (2023-07-06T20:30:35Z)
• AmbieGen: A Search-based Framework for Autonomous Systems Testing [12.93632948681342]
  AmbieGen is a search-based test case generation framework for autonomous systems. It supports test case generation for autonomous robots and autonomous car lane keeping assist systems.
  arXiv  Detail & Related papers  (2023-01-01T23:42:32Z)
• Open-Set Automatic Target Recognition [52.27048031302509]
  Automatic Target Recognition (ATR) is a category of computer vision algorithms which attempts to recognize targets on data obtained from different sensors. Existing ATR algorithms are developed for traditional closed-set methods where training and testing have the same class distribution. We propose an Open-set Automatic Target Recognition framework where we enable open-set recognition capability for ATR algorithms.
  arXiv  Detail & Related papers  (2022-11-10T21:28:24Z)
• Prompt-driven efficient Open-set Semi-supervised Learning [52.30303262499391]
  Open-set semi-supervised learning (OSSL) has attracted growing interest, which investigates a more practical scenario where out-of-distribution (OOD) samples are only contained in unlabeled data. We propose a prompt-driven efficient OSSL framework, called OpenPrompt, which can propagate class information from labeled to unlabeled data with only a small number of trainable parameters.
  arXiv  Detail & Related papers  (2022-09-28T16:25:08Z)
• A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation [26.10081199009559]
  Testing the system on the road is the closest to real-world and desirable approach, but it is incredibly costly. A popular alternative is to evaluate an ADS's performance in some well-designed challenging scenarios, a.k.a. scenario-based testing. High-fidelity simulators have been widely used in this setting to maximize flexibility and convenience in testing what-if scenarios.
  arXiv  Detail & Related papers  (2021-12-02T03:41:33Z)
• Data Driven Testing of Cyber Physical Systems [12.93632948681342]
  We propose an approach to automatically generate fault-revealing test cases for CPS. Data collected from an application managing a smart building have been used to learn models of the environment.
  arXiv  Detail & Related papers  (2021-02-23T04:55:10Z)
• PyGlove: Symbolic Programming for Automated Machine Learning [88.15565138144042]
  We introduce a new way of programming AutoML based on symbolic programming. Under this paradigm, ML programs are mutable, thus can be manipulated easily by another program. We show that PyGlove users can easily convert a static program into a search space, quickly iterate on the search spaces and search algorithms, and craft complex search flows.
  arXiv  Detail & Related papers  (2021-01-21T19:05:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.