Towards a Complete Metamorphic Testing Pipeline
- URL: http://arxiv.org/abs/2310.00338v1
- Date: Sat, 30 Sep 2023 10:49:22 GMT
- Title: Towards a Complete Metamorphic Testing Pipeline
- Authors: Alejandra Duque-Torres, Dietmar Pfahl
- Abstract summary: Metamorphic Testing (MT) addresses the test oracle problem by examining the relationships between input-output pairs in consecutive executions of the System Under Test (SUT)
These relations, known as Metamorphic Relations (MRs), specify the expected output changes resulting from specific input changes.
Our research aims to develop methods and tools that assist testers in generating MRs, defining constraints, and providing explainability for MR outcomes.
- Score: 56.75969180129005
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Metamorphic Testing (MT) addresses the test oracle problem by examining the
relationships between input-output pairs in consecutive executions of the
System Under Test (SUT). These relations, known as Metamorphic Relations (MRs),
specify the expected output changes resulting from specific input changes.
However, achieving full automation in generating, selecting, and understanding
MR violations poses challenges. Our research aims to develop methods and tools
that assist testers in generating MRs, defining constraints, and providing
explainability for MR outcomes. In the MR generation phase, we explore
automated techniques that utilise a domain-specific language to generate and
describe MRs. The MR constraint definition focuses on capturing the nuances of
MR applicability by defining constraints. These constraints help identify the
specific conditions under which MRs are expected to hold. The evaluation and
validation involve conducting empirical studies to assess the effectiveness of
the developed methods and validate their applicability in real-world regression
testing scenarios. Through this research, we aim to advance the automation of
MR generation, enhance the understanding of MR violations, and facilitate their
effective application in regression testing.
Related papers
- Towards Generating Executable Metamorphic Relations Using Large Language Models [46.26208489175692]
We propose an approach for automatically deriving executable MRs from requirements using large language models (LLMs)
To assess the feasibility of our approach, we conducted a questionnaire-based survey in collaboration with Siemens Industry Software.
arXiv Detail & Related papers (2024-01-30T13:52:47Z) - Self-RAG: Learning to Retrieve, Generate, and Critique through
Self-Reflection [74.51523859064802]
We introduce a new framework called Self-Reflective Retrieval-Augmented Generation (Self-RAG)
Self-RAG enhances an LM's quality and factuality through retrieval and self-reflection.
It significantly outperforms state-of-the-art LLMs and retrieval-augmented models on a diverse set of tasks.
arXiv Detail & Related papers (2023-10-17T18:18:32Z) - Ensemble Differential Evolution with Simulation-Based Hybridization and
Self-Adaptation for Inventory Management Under Uncertainty [0.0]
This study proposes an Ensemble Differential Evolution with Simula-tion-Based Hybridization and Self-Adaptation (EDESH-SA) approach for inven-tory management (IM) under uncertainty.
arXiv Detail & Related papers (2023-09-22T13:25:58Z) - Exploring a Test Data-Driven Method for Selecting and Constraining
Metamorphic Relations [46.889513596156185]
This paper presents a preliminary evaluation of MetaTrimmer, a method for selecting and constraining Metamorphic Relations based on test data.
The novelty of MetaTrimmer is its avoidance of complex prediction models that require labeled datasets regarding the applicability of MRs.
In a preliminary evaluation, MetaTrimmer shows the potential to overcome existing limitations and enhance MR effectiveness.
arXiv Detail & Related papers (2023-07-28T12:27:34Z) - MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases [9.00297842984345]
We propose MR-Scout to automatically synthesize MRs from test cases in open-source software projects.
Over 97% of codified MRs are of high quality for automated test case generation.
Our qualitative study shows that 55.76% to 76.92% of codified MRs are easily comprehensible for developers.
arXiv Detail & Related papers (2023-04-15T12:53:32Z) - Reinforcement Learning with a Terminator [80.34572413850186]
We learn the parameters of the TerMDP and leverage the structure of the estimation problem to provide state-wise confidence bounds.
We use these to construct a provably-efficient algorithm, which accounts for termination, and bound its regret.
arXiv Detail & Related papers (2022-05-30T18:40:28Z) - Effective Mutation Rate Adaptation through Group Elite Selection [50.88204196504888]
This paper introduces the Group Elite Selection of Mutation Rates (GESMR) algorithm.
GESMR co-evolves a population of solutions and a population of MRs, such that each MR is assigned to a group of solutions.
With the same number of function evaluations and with almost no overhead, GESMR converges faster and to better solutions than previous approaches.
arXiv Detail & Related papers (2022-04-11T01:08:26Z) - Metamorphic Relation Prioritization for Effective Regression Testing [11.033596835816422]
We propose approaches to prioritize metamorphic relations (MRs) to improve the efficiency and effectiveness of regression testing.
We present two MR prioritization approaches: (1) fault-based and (2) coverage-based.
Our results show that fault-based MR prioritization leads to reducing the number of source and follow-up test cases that needs to be executed.
arXiv Detail & Related papers (2021-09-20T19:06:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.