Prioritized Variable-length Test Cases Generation for Finite State
Machines
- URL: http://arxiv.org/abs/2203.09596v1
- Date: Thu, 17 Mar 2022 20:16:45 GMT
- Title: Prioritized Variable-length Test Cases Generation for Finite State
Machines
- Authors: Vaclav Rechtberger, Miroslav Bures, Bestoun S. Ahmed, Youcef Belkhier,
Jiri Nema, Hynek Schvach
- Abstract summary: Model-based Testing (MBT) is an effective approach for testing when parts of a system-under-test have the characteristics of a finite state machine (FSM)
This paper presents a test generation strategy that satisfies all these requirements.
Depending on the application of the FSM, the strategy and evaluation presented in this paper are applicable both in testing functional and non-functional software requirements.
- Score: 0.09786690381850353
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Model-based Testing (MBT) is an effective approach for testing when parts of
a system-under-test have the characteristics of a finite state machine (FSM).
Despite various strategies in the literature on this topic, little work exists
to handle special testing situations. More specifically, when concurrently: (1)
the test paths can start and end only in defined states of the FSM, (2) a
prioritization mechanism that requires only defined states and transitions of
the FSM to be visited by test cases is required, and (3) the test paths must be
in a given length range, not necessarily of explicit uniform length. This paper
presents a test generation strategy that satisfies all these requirements. A
concurrent combination of these requirements is highly practical for real
industrial testing. Six variants of possible algorithms to implement this
strategy are described. Using a mixture of 180 problem instances from real
automotive and defense projects and artificially generated FSMs, all variants
are compared with a baseline strategy based on an established N-switch coverage
concept modification. Various properties of the generated test paths and their
potential to activate fictional defects defined in FSMs are evaluated. The
presented strategy outperforms the baseline in most problem configurations. Out
of the six analyzed variants, three give the best results even though a
universal best performer is hard to identify. Depending on the application of
the FSM, the strategy and evaluation presented in this paper are applicable
both in testing functional and non-functional software requirements.
Related papers
- UniTTA: Unified Benchmark and Versatile Framework Towards Realistic Test-Time Adaptation [66.05528698010697]
Test-Time Adaptation aims to adapt pre-trained models to the target domain during testing.
Researchers have identified various challenging scenarios and developed diverse methods to address these challenges.
We propose a Unified Test-Time Adaptation benchmark, which is comprehensive and widely applicable.
arXiv Detail & Related papers (2024-07-29T15:04:53Z) - Realistic Evaluation of Test-Time Adaptation Algorithms: Unsupervised Hyperparameter Selection [1.4530711901349282]
Test-Time Adaptation (TTA) has emerged as a promising strategy for tackling the problem of machine learning model robustness under distribution shifts.
We evaluate existing TTA methods using surrogate-based hp-selection strategies to obtain a more realistic evaluation of their performance.
arXiv Detail & Related papers (2024-07-19T11:58:30Z) - Fuzzy Inference System for Test Case Prioritization in Software Testing [0.0]
Test case prioritization ( TCP) is a vital strategy to enhance testing efficiency.
This paper introduces a novel fuzzy logic-based approach to automate TCP.
arXiv Detail & Related papers (2024-04-25T08:08:54Z) - Generating Test Scenarios from NL Requirements using Retrieval-Augmented LLMs: An Industrial Study [5.179738379203527]
This paper presents an automated approach (RAGTAG) for test scenario generation using Retrieval-Augmented Generation (RAG) with Large Language Models (LLMs)
We evaluate RAGTAG on two industrial projects from Austrian Post with bilingual requirements in German and English.
arXiv Detail & Related papers (2024-04-19T10:27:40Z) - Test-Time Domain Generalization for Face Anti-Spoofing [60.94384914275116]
Face Anti-Spoofing (FAS) is pivotal in safeguarding facial recognition systems against presentation attacks.
We introduce a novel Test-Time Domain Generalization framework for FAS, which leverages the testing data to boost the model's generalizability.
Our method, consisting of Test-Time Style Projection (TTSP) and Diverse Style Shifts Simulation (DSSS), effectively projects the unseen data to the seen domain space.
arXiv Detail & Related papers (2024-03-28T11:50:23Z) - Deep anytime-valid hypothesis testing [29.273915933729057]
We propose a general framework for constructing powerful, sequential hypothesis tests for nonparametric testing problems.
We develop a principled approach of leveraging the representation capability of machine learning models within the testing-by-betting framework.
Empirical results on synthetic and real-world datasets demonstrate that tests instantiated using our general framework are competitive against specialized baselines.
arXiv Detail & Related papers (2023-10-30T09:46:19Z) - Generative Judge for Evaluating Alignment [84.09815387884753]
We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges.
Our model is trained on user queries and LLM-generated responses under massive real-world scenarios.
Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
arXiv Detail & Related papers (2023-10-09T07:27:15Z) - On Pitfalls of Test-Time Adaptation [82.8392232222119]
Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts.
We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
arXiv Detail & Related papers (2023-06-06T09:35:29Z) - MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation [104.48766162008815]
We propose and explore a new multi-modal extension of test-time adaptation for 3D semantic segmentation.
To design a framework that can take full advantage of multi-modality, each modality provides regularized self-supervisory signals to other modalities.
Our regularized pseudo labels produce stable self-learning signals in numerous multi-modal test-time adaptation scenarios.
arXiv Detail & Related papers (2022-04-27T02:28:12Z) - Overview of Test Coverage Criteria for Test Case Generation from Finite
State Machines Modelled as Directed Graphs [0.12891210250935145]
Test Coverage criteria are an essential concept for test engineers when generating the test cases from a System Under Test model.
Test Coverage criteria define the number of actions or combinations by which a system is tested.
This study summarized all commonly used test coverage criteria for Finite State Machines and discussed them regarding their subsumption, equivalence, or non-comparability.
arXiv Detail & Related papers (2022-03-17T20:30:14Z) - Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results.
Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples.
Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
arXiv Detail & Related papers (2020-08-04T10:41:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.