Measuring Software Testability via Automatically Generated Test Cases
- URL: http://arxiv.org/abs/2307.16185v1
- Date: Sun, 30 Jul 2023 09:48:51 GMT
- Title: Measuring Software Testability via Automatically Generated Test Cases
- Authors: Luca Guglielmo, Leonardo Mariani, Giovanni Denaro
- Abstract summary: We propose a new approach to pursuing testability measurements based on software metrics.
Our approach exploits automatic test generation and mutation analysis to quantify the evidence about the relative hardness of developing effective test cases.
- Score: 8.17364116624769
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Estimating software testability can crucially assist software managers to
optimize test budgets and software quality. In this paper, we propose a new
approach that radically differs from the traditional approach of pursuing
testability measurements based on software metrics, e.g., the size of the code
or the complexity of the designs. Our approach exploits automatic test
generation and mutation analysis to quantify the evidence about the relative
hardness of developing effective test cases. In the paper, we elaborate on the
intuitions and the methodological choices that underlie our proposal for
estimating testability, introduce a technique and a prototype that allows for
concretely estimating testability accordingly, and discuss our findings out of
a set of experiments in which we compare the performance of our estimations
both against and in combination with traditional software metrics. The results
show that our testability estimates capture a complementary dimension of
testability that can be synergistically combined with approaches based on
software metrics to improve the accuracy of predictions.
Related papers
- Fuzzy Inference System for Test Case Prioritization in Software Testing [0.0]
Test case prioritization ( TCP) is a vital strategy to enhance testing efficiency.
This paper introduces a novel fuzzy logic-based approach to automate TCP.
arXiv Detail & Related papers (2024-04-25T08:08:54Z) - InterEvo-TR: Interactive Evolutionary Test Generation With Readability
Assessment [1.6874375111244329]
We propose incorporating interactive readability assessments made by a tester into EvoSuite.
Our approach, InterEvo-TR, interacts with the tester at different moments during the search.
Our results show that the strategy to select and present intermediate results is effective for the purpose of readability assessment.
arXiv Detail & Related papers (2024-01-13T13:14:29Z) - Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of
System-level Testing of Autonomous Vehicles [5.634825161148484]
We introduce a set of black-box test adequacy metrics called "Test suite Instance Space Adequacy" (TISA) metrics.
The TISA metrics offer a way to assess both the diversity and coverage of the test suite and the range of bugs detected during testing.
We evaluate the efficacy of the TISA metrics by examining their correlation with the number of bugs detected in system-level simulation testing of AVs.
arXiv Detail & Related papers (2023-11-14T10:16:05Z) - Test Case Recommendations with Distributed Representation of Code
Syntactic Features [2.225268436173329]
We propose an automated approach which exploits both structural and semantic properties of source code methods and test cases.
The proposed approach initially trains a neural network to transform method-level source code, as well as unit tests, into distributed representations.
The model computes cosine similarity between the method's embedding and the previously-embedded training instances.
arXiv Detail & Related papers (2023-10-04T21:42:01Z) - Validation Diagnostics for SBI algorithms based on Normalizing Flows [55.41644538483948]
This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF.
It also offers theoretical guarantees based on results of local consistency.
This work should help the design of better specified models or drive the development of novel SBI-algorithms.
arXiv Detail & Related papers (2022-11-17T15:48:06Z) - Uncertainty-Driven Action Quality Assessment [67.20617610820857]
We propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to capture the diversity among multiple judge scores.
We generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss.
Our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.
arXiv Detail & Related papers (2022-07-29T07:21:15Z) - Generalized Coverage Criteria for Combinatorial Sequence Testing [4.807321976136717]
We present a new model-based approach for testing systems that use sequences of actions and assertions as test vectors.
Our solution includes a method for quantifying testing quality, a tool for generating high-quality test suites based on the coverage criteria we propose, and a framework for assessing risks.
arXiv Detail & Related papers (2022-01-03T08:35:28Z) - Differential privacy and robust statistics in high dimensions [49.50869296871643]
High-dimensional Propose-Test-Release (HPTR) builds upon three crucial components: the exponential mechanism, robust statistics, and the Propose-Test-Release mechanism.
We show that HPTR nearly achieves the optimal sample complexity under several scenarios studied in the literature.
arXiv Detail & Related papers (2021-11-12T06:36:40Z) - Predictive machine learning for prescriptive applications: a coupled
training-validating approach [77.34726150561087]
We propose a new method for training predictive machine learning models for prescriptive applications.
This approach is based on tweaking the validation step in the standard training-validating-testing scheme.
Several experiments with synthetic data demonstrate promising results in reducing the prescription costs in both deterministic and real models.
arXiv Detail & Related papers (2021-10-22T15:03:20Z) - Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design.
We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
arXiv Detail & Related papers (2021-05-17T08:36:18Z) - SAMBA: Safe Model-Based & Active Reinforcement Learning [59.01424351231993]
SAMBA is a framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics.
We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations.
We provide intuition as to the effectiveness of the framework by a detailed analysis of our active metrics and safety constraints.
arXiv Detail & Related papers (2020-06-12T10:40:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.