The Impact of Defect (Re) Prediction on Software Testing

• URL: http://arxiv.org/abs/2404.11040v2
• Date: Tue, 10 Sep 2024 06:55:05 GMT
• Title: The Impact of Defect (Re) Prediction on Software Testing
• Authors: Yukasa Murakami, Yuta Yamasaki, Masateru Tsunoda, Akito Monden, Amjed Tahir, Kwabena Ebo Bennin, Koji Toda, Keitaro Nakasai,
• Abstract summary: Cross-project defect prediction (CPDP) aims to use data from external projects as historical data may not be available from the same project. A Bandit Algorithm (BA) based approach has been proposed in prior research to select the most suitable learning project. This study aims to improve the BA method to reduce defects overlooking, especially during the early testing stages.
• Score: 1.5869998695491834
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-project defect prediction (CPDP) aims to use data from external projects as historical data may not be available from the same project. In CPDP, deciding on a particular historical project to build a training model can be difficult. To help with this decision, a Bandit Algorithm (BA) based approach has been proposed in prior research to select the most suitable learning project. However, this BA method could lead to the selection of unsuitable data during the early iteration of BA (i.e., early stage of software testing). Selecting an unsuitable model can reduce the prediction accuracy, leading to potential defect overlooking. This study aims to improve the BA method to reduce defects overlooking, especially during the early testing stages. Once all modules have been tested, modules tested in the early stage are re-predicted, and some modules are retested based on the re-prediction. To assess the impact of re-prediction and retesting, we applied five kinds of BA methods, using 8, 16, and 32 OSS projects as learning data. The results show that the newly proposed approach steadily reduced the probability of defect overlooking without degradation of prediction accuracy.

Related papers

• A Probabilistic Perspective on Unlearning and Alignment for Large Language Models [48.96686419141881]
  We introduce the first formal probabilistic evaluation framework in Large Language Models (LLMs) We derive novel metrics with high-probability guarantees concerning the output distribution of a model. Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment.
  arXiv  Detail & Related papers  (2024-10-04T15:44:23Z)
• An Empirical Study of the Impact of Test Strategies on Online Optimization for Ensemble-Learning Defect Prediction [2.547631669143471]
  We employ bandit algorithms (BA), an online optimization method, to select the highest-accuracy ensemble method. We used six popular defect prediction datasets, four ensemble learning methods such as bagging, and three test strategies such as testing positive-prediction modules first (PF) Our results show that when BA is applied with PF, the prediction accuracy improved on average, and the number of found defects increased by 7% on a minimum of five out of six datasets.
  arXiv  Detail & Related papers  (2024-09-10T07:06:50Z)
• Building Defect Prediction Models by Online Learning Considering Defect Overlooking [1.5869998695491834]
  Building defect prediction models based on online learning can enhance prediction accuracy. A module predicted as "non-defective" can result in fewer test cases for such modules. erroneous test results are used as learning data by online learning, which could negatively affect prediction accuracy.
  arXiv  Detail & Related papers  (2024-04-17T03:20:46Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications. We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale [53.152460508207184]
  Source-Free Unsupervised Domain Adaptation (SFUDA) is a challenging task where a model needs to be adapted to a new domain without access to target domain labels or source domain data. This paper proposes a novel approach that considers multiple prediction hypotheses for each sample and investigates the rationale behind each hypothesis. To achieve the optimal performance, we propose a three-step adaptation process: model pre-adaptation, hypothesis consolidation, and semi-supervised learning.
  arXiv  Detail & Related papers  (2024-02-02T05:53:22Z)
• Software Defect Prediction by Online Learning Considering Defect Overlooking [1.655352281097533]
  Building defect prediction models based on online learning can enhance prediction accuracy. It continuously rebuilds a new prediction model when adding a new data point. However, predicting a module as "non-defective" (i.e., negative prediction) can result in fewer test cases for such modules.
  arXiv  Detail & Related papers  (2023-08-25T15:02:22Z)
• Learning Sample Difficulty from Pre-trained Models for Reliable Prediction [55.77136037458667]
  We propose to utilize large-scale pre-trained models to guide downstream model training with sample difficulty-aware entropy regularization. We simultaneously improve accuracy and uncertainty calibration across challenging benchmarks.
  arXiv  Detail & Related papers  (2023-04-20T07:29:23Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Moving from Cross-Project Defect Prediction to Heterogeneous Defect Prediction: A Partial Replication Study [0.0]
  Earlier studies often used machine learning techniques to build, validate, and improve bug prediction models. Knowledge coming from those models will not be overlapping to a target project if no sufficient metrics have been collected in the source projects. We systematically integrated Heterogeneous Defect Prediction (HDP) by replicating and validating the obtained results. Our results shed light on the infeasibility of many cases for the HDP algorithm due to its sensitivity to the parameter selection.
  arXiv  Detail & Related papers  (2021-03-05T06:29:45Z)
• Counterfactual Predictions under Runtime Confounding [74.90756694584839]
  We study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data. We propose a doubly-robust procedure for learning counterfactual prediction models in this setting.
  arXiv  Detail & Related papers  (2020-06-30T15:49:05Z)
• Software Defect Prediction Based On Deep Learning Models: Performance Study [0.5735035463793008]
  Two deep learning models, Stack Sparse Auto-Encoder (SSAE) and Deep Belief Network (DBN) are deployed to classify NASA datasets. According to the conducted experiment, the accuracy for the datasets with sufficient samples is enhanced.
  arXiv  Detail & Related papers  (2020-04-02T06:02:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.