Testing CPS with Design Assumptions-Based Metamorphic Relations and Genetic Programming

• URL: http://arxiv.org/abs/2412.03330v1
• Date: Wed, 04 Dec 2024 13:59:48 GMT
• Title: Testing CPS with Design Assumptions-Based Metamorphic Relations and Genetic Programming
• Authors: Claudio Mandrioli, Seung Yeob Shin, Domenico Bianculli, Lionel Briand,
• Abstract summary: Cyber-Physical Systems (CPSs) software is used to enforce desired behaviours on physical systems. CPS requirements describe how closely the actual physical traces should track the desired traces. Our work proposes an approach to testing CPS using control-theoretical design assumptions instead of requirements.
• Score: 2.195923771201972
• License:
• Abstract: Cyber-Physical Systems (CPSs) software is used to enforce desired behaviours on physical systems. To test the interaction between the CPS software and the system's physics, engineers provide traces of desired physical states and observe traces of the actual physical states. CPS requirements describe how closely the actual physical traces should track the desired traces. These requirements are typically defined for specific, simple input traces such as step or ramp sequences, and thus are not applicable to arbitrary inputs. This limits the availability of oracles for CPSs. Our recent work proposes an approach to testing CPS using control-theoretical design assumptions instead of requirements. This approach circumvents the oracle problem by leveraging the control-theoretical guarantees that are provided when the design assumptions are satisfied. To address the test case generation and oracle problems, researchers have proposed metamorphic testing, which is based on the study of relations across tests, i.e., metamorphic relations (MRs). In this work, we define MRs based on the design assumptions and explore combinations of these MRs using genetic programming to generate CPS test cases. This enables the generation of CPS input traces with potentially arbitrary shapes, together with associated expected output traces. We use the deviation from the expected output traces to guide the generation of input traces that falsify the MRs. Our experiment results show that the MR-falsification provides engineers with new information, helping them identify passed and failed test cases. Furthermore, we show that the generation of traces that falsify the MRs is a non-trivial problem, which is successfully addressed by our genetic search.

Related papers

• LLM-assisted Physical Invariant Extraction for Cyber-Physical Systems Anomaly Detection [2.276945627589248]
  Cyber-Physical Systems (CPS) are vulnerable to cyber-attacks with potentially catastrophic effects. CPS design documentation often contains semantically rich descriptions of physical procedures. We propose a novel approach to extract physical invariants from CPS testbeds for anomaly detection.
  arXiv  Detail & Related papers  (2024-11-17T00:09:04Z)
• Tractable Offline Learning of Regular Decision Processes [50.11277112628193]
  This work studies offline Reinforcement Learning (RL) in a class of non-Markovian environments called Regular Decision Processes (RDPs) Ins, the unknown dependency of future observations and rewards from the past interactions can be captured experimentally. Many algorithms first reconstruct this unknown dependency using automata learning techniques.
  arXiv  Detail & Related papers  (2024-09-04T14:26:58Z)
• Nonparametric independence tests in high-dimensional settings, with applications to the genetics of complex disease [55.2480439325792]
  We show how defining adequate premetric structures on the support spaces of the genetic data allows for novel approaches to such testing. For each problem, we provide mathematical results, simulations and the application to real data.
  arXiv  Detail & Related papers  (2024-07-29T01:00:53Z)
• Collaborative non-parametric two-sample testing [55.98760097296213]
  The goal is to identify nodes where the null hypothesis $p_v = q_v$ should be rejected. We propose the non-parametric collaborative two-sample testing (CTST) framework that efficiently leverages the graph structure. Our methodology integrates elements from f-divergence estimation, Kernel Methods, and Multitask Learning.
  arXiv  Detail & Related papers  (2024-02-08T14:43:56Z)
• Towards a Complete Metamorphic Testing Pipeline [56.75969180129005]
  Metamorphic Testing (MT) addresses the test oracle problem by examining the relationships between input-output pairs in consecutive executions of the System Under Test (SUT) These relations, known as Metamorphic Relations (MRs), specify the expected output changes resulting from specific input changes. Our research aims to develop methods and tools that assist testers in generating MRs, defining constraints, and providing explainability for MR outcomes.
  arXiv  Detail & Related papers  (2023-09-30T10:49:22Z)
• Test Case Generation and Test Oracle Support for Testing CPSs using Hybrid Models [2.6166087473624313]
  Cyber-Physical Systems (CPSs) play a central role in the behavior of a wide range of autonomous physical systems. CPSs are often specified iteratively as a sequence of models at different levels that can be tested via simulation systems. One such model is a hybrid automaton; these are used frequently for CPS applications and have the advantage of encapsulating both continuous and discrete CPS behaviors.
  arXiv  Detail & Related papers  (2023-09-14T19:08:09Z)
• Stress Testing Control Loops in Cyber-Physical Systems [2.195923771201972]
  We investigate the testing of control-based CPSs, where control and software engineers develop the software collaboratively. We define stress testing of control-based CPSs as generating tests to falsify such design assumptions. We evaluate our approach on three case study systems, including a drone, a continuous-current motor, and an aircraft.
  arXiv  Detail & Related papers  (2023-02-27T16:01:38Z)
• MTGP: Combining Metamorphic Testing and Genetic Programming [1.6058099298620425]
  Genetic programming is not yet mature enough for a practical use in real-world software development. We suggest MTGP, which combines metamorphic testing and genetic programming. We find that using metamorphic testing combined with labeled training cases leads to a higher generalization rate.
  arXiv  Detail & Related papers  (2023-01-20T16:35:22Z)
• Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling [77.15766509677348]
  Conditional generative models often inherit spurious correlations from the training dataset. This can result in label-conditional distributions that are imbalanced with respect to another latent attribute. We propose a general two-step strategy to mitigate this issue.
  arXiv  Detail & Related papers  (2022-12-05T08:09:33Z)
• CoCoMoT: Conformance Checking of Multi-Perspective Processes via SMT (Extended Version) [62.96267257163426]
  We introduce the CoCoMoT (Computing Conformance Modulo Theories) framework. First, we show how SAT-based encodings studied in the pure control-flow setting can be lifted to our data-aware case. Second, we introduce a novel preprocessing technique based on a notion of property-preserving clustering.
  arXiv  Detail & Related papers  (2021-03-18T20:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.