MOTIF: A tool for Mutation Testing with Fuzzing

• URL: http://arxiv.org/abs/2406.02398v1
• Date: Tue, 4 Jun 2024 15:12:01 GMT
• Title: MOTIF: A tool for Mutation Testing with Fuzzing
• Authors: Jaekwon Lee, Enrico Viganò, Fabrizio Pastore, Lionel Briand,
• Abstract summary: Mutation testing is a desired practice for embedded software running in safety-critical cyber-physical systems. MOTIF overcomes limitations by leveraging grey-box fuzzing tools to generate unit test cases in C that detect injected faults in mutants.
• Score: 3.4742750855568763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mutation testing consists of generating test cases that detect faults injected into software (generating mutants) which its original test suite could not. By running such an augmented set of test cases, it may discover actual faults that may have gone unnoticed with the original test suite. It is thus a desired practice for embedded software running in safety-critical cyber-physical systems (CPS). Unfortunately, the state-of-the-art tool targeting C, a typical language for CPS software, relies on symbolic execution, whose limitations often prevent its application. MOTIF overcomes such limitations by leveraging grey-box fuzzing tools to generate unit test cases in C that detect injected faults in mutants. Indeed, fuzzing tools automatically generate inputs by exercising the compiled version of the software under test guided by coverage feedback, thus overcoming the limitations of symbolic execution. Our empirical assessment has shown that it detects more faults than symbolic execution (i.e., up to 47 percentage points), when the latter is applicable.

Related papers

• The Impact of SBOM Generators on Vulnerability Assessment in Python: A Comparison and a Novel Approach [56.4040698609393]
  Software Bill of Materials (SBOM) has been promoted as a tool to increase transparency and verifiability in software composition. Current SBOM generation tools often suffer from inaccuracies in identifying components and dependencies. We propose PIP-sbom, a novel pip-inspired solution that addresses their shortcomings.
  arXiv  Detail & Related papers  (2024-09-10T10:12:37Z)
• Automatic Generation of Test Cases based on Bug Reports: a Feasibility Study with Large Language Models [4.318319522015101]
  Existing approaches produce test cases that either can be qualified as simple (e.g. unit tests) or that require precise specifications. Most testing procedures still rely on test cases written by humans to form test suites. We investigate the feasibility of performing this generation by leveraging large language models (LLMs) and using bug reports as inputs.
  arXiv  Detail & Related papers  (2023-10-10T05:30:12Z)
• Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing [13.743062498008555]
  We introduce MuTAP for improving the effectiveness of test cases generated by Large Language Models (LLMs) in terms of revealing bugs. MuTAP is capable of generating effective test cases in the absence of natural language descriptions of the Program Under Test (PUTs) Our results show that our proposed method is able to detect up to 28% more faulty human-written code snippets.
  arXiv  Detail & Related papers  (2023-08-31T08:48:31Z)
• Fuzzing for CPS Mutation Testing [3.512722797771289]
  We propose a mutation testing approach that leverages fuzz testing, which has proved effective with C and C++ software. Our empirical evaluation shows that mutation testing based on fuzz testing kills a significantly higher proportion of live mutants than symbolic execution.
  arXiv  Detail & Related papers  (2023-08-15T16:35:31Z)
• Towards Automatic Generation of Amplified Regression Test Oracles [44.45138073080198]
  We propose a test oracle derivation approach to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour.
  arXiv  Detail & Related papers  (2023-07-28T12:38:44Z)
• CodeT: Code Generation with Generated Tests [49.622590050797236]
  We explore the use of pre-trained language models to automatically generate test cases. CodeT executes the code solutions using the generated test cases, and then chooses the best solution. We evaluate CodeT on five different pre-trained models with both HumanEval and MBPP benchmarks.
  arXiv  Detail & Related papers  (2022-07-21T10:18:37Z)
• Fault-Aware Neural Code Rankers [64.41888054066861]
  We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it. Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
  arXiv  Detail & Related papers  (2022-06-04T22:01:05Z)
• SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video Games Using Risk Based Testing and Machine Learning [62.997667081978825]
  Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems. We present SUPERNOVA, a system responsible for test selection and defect prevention while also functioning as an automation hub. The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title.
  arXiv  Detail & Related papers  (2022-03-10T00:47:46Z)
• DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score [4.444652484439581]
  tool is effective at augmenting the given test set, increasing its capability to detect mutants by 63% on average. A leave-one-out experiment shows that the augmented test set is capable of exposing unseen mutants.
  arXiv  Detail & Related papers  (2021-09-15T18:20:50Z)
• Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection [67.53296659361598]
  adversarial EXEmples can bypass machine learning-based detection by perturbing relatively few input bytes. We develop a unifying framework that does not only encompass and generalize previous attacks against machine-learning models, but also includes three novel attacks. These attacks, named Full DOS, Extend and Shift, inject the adversarial payload by respectively manipulating the DOS header, extending it, and shifting the content of the first section.
  arXiv  Detail & Related papers  (2020-08-17T07:16:57Z)
• Detection of Coincidentally Correct Test Cases through Random Forests [1.2891210250935143]
  We propose a hybrid approach of ensemble learning combined with a supervised learning algorithm namely, Random Forests (RF) for the purpose of correctly identifying test cases that are mislabeled to be the passing test cases. A cost-effective analysis of flipping the test status or trimming (i.e., eliminating from the computation) the coincidental correct test cases is also reported.
  arXiv  Detail & Related papers  (2020-06-14T15:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.