An Empirical Evaluation of Manually Created Equivalent Mutants

• URL: http://arxiv.org/abs/2404.09241v1
• Date: Sun, 14 Apr 2024 13:04:10 GMT
• Title: An Empirical Evaluation of Manually Created Equivalent Mutants
• Authors: Philipp Straubinger, Alexander Degenhart, Gordon Fraser,
• Abstract summary: Less than 10 % of manually created mutants are equivalent. Surprisingly, our findings indicate that a significant portion of developers struggle to accurately identify equivalent mutants.
• Score: 54.02049952279685
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Mutation testing consists of evaluating how effective test suites are at detecting artificially seeded defects in the source code, and guiding the improvement of the test suites. Although mutation testing tools are increasingly adopted in practice, equivalent mutants, i.e., mutants that differ only in syntax but not semantics, hamper this process. While prior research investigated how frequently equivalent mutants are produced by mutation testing tools and how effective existing methods of detecting these equivalent mutants are, it remains unclear to what degree humans also create equivalent mutants, and how well they perform at identifying these. We therefore study these questions in the context of Code Defenders, a mutation testing game, in which players competitively produce mutants and tests. Using manual inspection as well as automated identification methods we establish that less than 10 % of manually created mutants are equivalent. Surprisingly, our findings indicate that a significant portion of developers struggle to accurately identify equivalent mutants, emphasizing the need for improved detection mechanisms and developer training in mutation testing.

Related papers

• Learning to Predict Mutation Effects of Protein-Protein Interactions by Microenvironment-aware Hierarchical Prompt Learning [78.38442423223832]
  We develop a novel codebook pre-training task, namely masked microenvironment modeling. We demonstrate superior performance and training efficiency over state-of-the-art pre-training-based methods in mutation effect prediction.
  arXiv  Detail & Related papers  (2024-05-16T03:53:21Z)
• Cal-DETR: Calibrated Detection Transformer [67.75361289429013]
  We propose a mechanism for calibrated detection transformers (Cal-DETR), particularly for Deformable-DETR, UP-DETR and DINO. We develop an uncertainty-guided logit modulation mechanism that leverages the uncertainty to modulate the class logits. Results corroborate the effectiveness of Cal-DETR against the competing train-time methods in calibrating both in-domain and out-domain detections.
  arXiv  Detail & Related papers  (2023-11-06T22:13:10Z)
• Binary Quantification and Dataset Shift: An Experimental Investigation [54.14283123210872]
  Quantification is the supervised learning task that consists of training predictors of the class prevalence values of sets of unlabelled data. The relationship between quantification and other types of dataset shift remains, by and large, unexplored. We propose a fine-grained taxonomy of types of dataset shift, by establishing protocols for the generation of datasets affected by these types of shift.
  arXiv  Detail & Related papers  (2023-10-06T20:11:27Z)
• Contextual Predictive Mutation Testing [17.832774161583036]
  We introduce MutationBERT, an approach for predictive mutation testing that simultaneously encodes the source method mutation and test method. Thanks to its higher precision, MutationBERT saves 33% of the time spent by a prior approach on checking/verifying live mutants. We validate our input representation, and aggregation approaches for lifting predictions from the test matrix level to the test suite level, finding similar improvements in performance.
  arXiv  Detail & Related papers  (2023-09-05T17:00:15Z)
• MuRS: Mutant Ranking and Suppression using Identifier Templates [4.9205581820379765]
  Google's mutation testing service integrates diff-based mutation testing into the code review process. Google's mutation testing service implements a number of suppression rules, which target not-useful mutants. This paper proposes and evaluates MuRS, an automated approach that groups mutants by patterns in the source code under test.
  arXiv  Detail & Related papers  (2023-06-15T13:43:52Z)
• Systematic Assessment of Fuzzers using Mutation Analysis [20.91546707828316]
  In software testing, the gold standard for evaluating test quality is mutation analysis. mutation analysis subsumes various coverage measures and provides a large and diverse set of faults. We apply modern mutation analysis techniques that pool multiple mutations and allow us -- for the first time -- to evaluate and compare fuzzers with mutation analysis.
  arXiv  Detail & Related papers  (2022-12-06T15:47:47Z)
• Robust Calibration with Multi-domain Temperature Scaling [86.07299013396059]
  We develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains. Our proposed method -- multi-domain temperature scaling -- uses the robustness in the domains to improve calibration under distribution shift.
  arXiv  Detail & Related papers  (2022-06-06T17:32:12Z)
• MutFormer: A context-dependent transformer-based model to predict pathogenic missense mutations [5.153619184788929]
  missense mutations account for approximately half of the known variants responsible for human inherited diseases. Recent advances in deep learning show that transformer models are particularly powerful at modeling sequences. We introduce MutFormer, a transformer-based model for prediction of pathogenic missense mutations.
  arXiv  Detail & Related papers  (2021-10-27T20:17:35Z)
• Robustness Verification for Transformers [165.25112192811764]
  We develop the first robustness verification algorithm for Transformers. The certified robustness bounds computed by our method are significantly tighter than those by naive Interval Bound propagation. These bounds also shed light on interpreting Transformers as they consistently reflect the importance of different words in sentiment analysis.
  arXiv  Detail & Related papers  (2020-02-16T17:16:31Z)
• DeepMutation: A Neural Mutation Tool [26.482720255691646]
  DeepMutation is a tool wrapping our deep learning model into a fully automated tool chain. It can generate, inject, and test mutants learned from real faults.
  arXiv  Detail & Related papers  (2020-02-12T01:57:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.