Test Case Generation and Test Oracle Support for Testing CPSs using Hybrid Models

• URL: http://arxiv.org/abs/2309.07994v1
• Date: Thu, 14 Sep 2023 19:08:09 GMT
• Title: Test Case Generation and Test Oracle Support for Testing CPSs using Hybrid Models
• Authors: Zahra Sadri-Moshkenani, Justin Bradley, Gregg Rothermel
• Abstract summary: Cyber-Physical Systems (CPSs) play a central role in the behavior of a wide range of autonomous physical systems. CPSs are often specified iteratively as a sequence of models at different levels that can be tested via simulation systems. One such model is a hybrid automaton; these are used frequently for CPS applications and have the advantage of encapsulating both continuous and discrete CPS behaviors.
• Score: 2.6166087473624313
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Cyber-Physical Systems (CPSs) play a central role in the behavior of a wide range of autonomous physical systems such as medical devices, autonomous vehicles, and smart homes, many of which are safety-critical. CPSs are often specified iteratively as a sequence of models at different levels that can be tested via simulation systems at early stages of their development cycle. One such model is a hybrid automaton; these are used frequently for CPS applications and have the advantage of encapsulating both continuous and discrete CPS behaviors. When testing CPSs, engineers can take advantage of these models to generate test cases that target both types of these behaviors. Moreover, since these models are constructed early in the development process for CPSs, they allow test cases to be generated early in that process for those CPSs, even before simulation models of the CPSs have been designed. One challenge when testing CPSs is that these systems may operate differently even under an identically applied test scenario. In such cases, we cannot employ test oracles that use predetermined deterministic behaviors; instead, test oracles should consider sets of desired behaviors in order to determine whether the CPS has behaved appropriately. In this paper we present a test case generation technique, HYTEST, that generates test cases based on hybrid models, accompanied by appropriate test oracles, for use in testing CPSs early in their development cycle. To evaluate the effectiveness and efficiency of HYTEST, we conducted an empirical study in which we applied the technique to several CPSs and measured its ability to detect faults in those CPSs and the amount of time required to perform the testing process. The results of the study show that HYTEST was able to detect faults more effectively and efficiently than the baseline techniques we compare it to.

Related papers

• Testing CPS with Design Assumptions-Based Metamorphic Relations and Genetic Programming [2.195923771201972]
  Cyber-Physical Systems (CPSs) software is used to enforce desired behaviours on physical systems. CPS requirements describe how closely the actual physical traces should track the desired traces. Our work proposes an approach to testing CPS using control-theoretical design assumptions instead of requirements.
  arXiv  Detail & Related papers  (2024-12-04T13:59:48Z)
• Context-Aware Fuzzing for Robustness Enhancement of Deep Learning Models [3.4790378315775348]
  We propose a novel testing metric called Contextual Confidence (CC) CC measures a test case through the surrounding samples of a test case in terms of their mean probability predicted to the prediction label of the test case. Clover computes the piece of difference between each such pair of a seed and a test case.
  arXiv  Detail & Related papers  (2024-07-17T09:31:10Z)
• Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study [61.64685376882383]
  Counterfactual learning to rank (CLTR) has attracted extensive attention in the IR community for its ability to leverage massive logged user interaction data to train ranking models. This paper investigates the robustness of existing CLTR models in complex and diverse situations. We find that the DLA models and IPS-DCM show better robustness under various simulation settings than IPS-PBM and PRS with offline propensity estimation.
  arXiv  Detail & Related papers  (2024-04-04T10:54:38Z)
• Robustness and Generalization Performance of Deep Learning Models on Cyber-Physical Systems: A Comparative Study [71.84852429039881]
  Investigation focuses on the models' ability to handle a range of perturbations, such as sensor faults and noise. We test the generalization and transfer learning capabilities of these models by exposing them to out-of-distribution (OOD) samples.
  arXiv  Detail & Related papers  (2023-06-13T12:43:59Z)
• On the Benefits of Semi-Supervised Test Case Generation for Simulation Models [16.28839850314951]
  Testing complex simulation models can be expensive and time consuming. Current state-of-the-art methods that explore this problem are fully-supervised. We introduce GenClu, which takes a semi-supervised approach.
  arXiv  Detail & Related papers  (2023-05-05T17:55:10Z)
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [117.72709110877939]
  Test-time adaptation (TTA) has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions. We categorize TTA into several distinct groups based on the form of test data, namely, test-time domain adaptation, test-time batch adaptation, and online test-time adaptation.
  arXiv  Detail & Related papers  (2023-03-27T16:32:21Z)
• Stress Testing Control Loops in Cyber-Physical Systems [2.195923771201972]
  We investigate the testing of control-based CPSs, where control and software engineers develop the software collaboratively. We define stress testing of control-based CPSs as generating tests to falsify such design assumptions. We evaluate our approach on three case study systems, including a drone, a continuous-current motor, and an aircraft.
  arXiv  Detail & Related papers  (2023-02-27T16:01:38Z)
• TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
  We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples. We show the success of our method on the common benchmark dataset CIFAR10-C.
  arXiv  Detail & Related papers  (2022-05-18T05:43:06Z)
• Data Driven Testing of Cyber Physical Systems [12.93632948681342]
  We propose an approach to automatically generate fault-revealing test cases for CPS. Data collected from an application managing a smart building have been used to learn models of the environment.
  arXiv  Detail & Related papers  (2021-02-23T04:55:10Z)
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design [63.48989885374238]
  When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testing algorithms that can operate in a noisy setting.
  arXiv  Detail & Related papers  (2020-04-26T23:41:33Z)
• Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework [68.96770035057716]
  A/B testing is a business strategy to compare a new product with an old one in pharmaceutical, technological, and traditional industries. This paper introduces a reinforcement learning framework for carrying A/B testing in online experiments.
  arXiv  Detail & Related papers  (2020-02-05T10:25:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.