Assessing test artifact quality -- A tertiary study
- URL: http://arxiv.org/abs/2402.09541v1
- Date: Wed, 14 Feb 2024 19:31:57 GMT
- Title: Assessing test artifact quality -- A tertiary study
- Authors: Huynh Khanh Vi Tran, Michael Unterkalmsteiner, J\"urgen B\"orstler,
Nauman bin Ali
- Abstract summary: We have carried out a systematic literature review to identify and analyze existing secondary studies on quality aspects of software testing artifacts.
We present an aggregation of the context dimensions and factors that can be used to characterize the environment in which the test case/suite quality is investigated.
- Score: 1.7827643249624088
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Context: Modern software development increasingly relies on software testing
for an ever more frequent delivery of high quality software. This puts high
demands on the quality of the central artifacts in software testing, test
suites and test cases. Objective: We aim to develop a comprehensive model for
capturing the dimensions of test case/suite quality, which are relevant for a
variety of perspectives. Method: We have carried out a systematic literature
review to identify and analyze existing secondary studies on quality aspects of
software testing artifacts. Results: We identified 49 relevant secondary
studies. Of these 49 studies, less than half did some form of quality appraisal
of the included primary studies and only 3 took into account the quality of the
primary study when synthesizing the results. We present an aggregation of the
context dimensions and factors that can be used to characterize the environment
in which the test case/suite quality is investigated. We also provide a
comprehensive model of test case/suite quality with definitions for the quality
attributes and measurements based on findings in the literature and ISO/IEC
25010:2011. Conclusion: The test artifact quality model presented in the paper
can be used to support test artifact quality assessment and improvement
initiatives in practice. Furtherm Information and Software Technology 139
(2021): 106620ore, the model can also be used as a framework for documenting
context characteristics to make research results more accessible for research
and practice.
Related papers
- Mashee at SemEval-2024 Task 8: The Impact of Samples Quality on the Performance of In-Context Learning for Machine Text Classification [0.0]
We employ the chi-square test to identify high-quality samples and compare the results with those obtained using low-quality samples.
Our findings demonstrate that utilizing high-quality samples leads to improved performance with respect to all evaluated metrics.
arXiv Detail & Related papers (2024-05-28T12:47:43Z) - QuRating: Selecting High-Quality Data for Training Language Models [64.83332850645074]
We introduce QuRating, a method for selecting pre-training data that can capture human intuitions about data quality.
In this paper, we investigate four qualities - writing style, required expertise, facts & trivia, and educational value.
We train a Qur model to learn scalar ratings from pairwise judgments, and use it to annotate a 260B training corpus with quality ratings for each of the four criteria.
arXiv Detail & Related papers (2024-02-15T06:36:07Z) - A manual categorization of new quality issues on automatically-generated
tests [0.8225289576465757]
We report on a manual analysis of an external dataset consisting of 2,340 automatically generated tests.
We propose a taxonomy of 13 new quality issues grouped in four categories.
We present eight recommendations that test generators may consider to improve the quality and usefulness of the automatically generated tests.
arXiv Detail & Related papers (2023-12-14T11:19:14Z) - A Novel Metric for Measuring Data Quality in Classification Applications
(extended version) [0.0]
We introduce and explain a novel metric to measure data quality.
This metric is based on the correlated evolution between the classification performance and the deterioration of data.
We provide an interpretation of each criterion and examples of assessment levels.
arXiv Detail & Related papers (2023-12-13T11:20:09Z) - Test-Case Quality -- Understanding Practitioners' Perspectives [1.7827643249624088]
We present a quality model which consists of 11 test-case quality attributes.
We identify a misalignment in defining test-case quality among practitioners and between academia and industry.
arXiv Detail & Related papers (2023-09-28T19:10:01Z) - Analyzing Dataset Annotation Quality Management in the Wild [63.07224587146207]
Even popular datasets used to train and evaluate state-of-the-art models contain a non-negligible amount of erroneous annotations, biases, or artifacts.
While practices and guidelines regarding dataset creation projects exist, large-scale analysis has yet to be performed on how quality management is conducted.
arXiv Detail & Related papers (2023-07-16T21:22:40Z) - Image Quality Assessment in the Modern Age [53.19271326110551]
This tutorial provides the audience with the basic theories, methodologies, and current progresses of image quality assessment (IQA)
We will first revisit several subjective quality assessment methodologies, with emphasis on how to properly select visual stimuli.
Both hand-engineered and (deep) learning-based methods will be covered.
arXiv Detail & Related papers (2021-10-19T02:38:46Z) - Implementation of Departmental and Periodical Examination Analyzer
System [0.0]
The Departmental and Periodical Examination System was developed using Visual Basic language.
The system was evaluated by a group of students, teachers, school administrators and information technology professionals.
arXiv Detail & Related papers (2021-03-09T06:47:20Z) - Quality meets Diversity: A Model-Agnostic Framework for Computerized
Adaptive Testing [60.38182654847399]
Computerized Adaptive Testing (CAT) is emerging as a promising testing application in many scenarios.
We propose a novel framework, namely Model-Agnostic Adaptive Testing (MAAT) for CAT solution.
arXiv Detail & Related papers (2021-01-15T06:48:50Z) - Generative Models are Unsupervised Predictors of Page Quality: A
Colossal-Scale Study [86.62171568318716]
Large generative language models such as GPT-2 are well-known for their ability to generate text.
We show that unsupervised predictors of "page quality" emerge, able to detect low quality content without any training.
We conduct extensive qualitative and quantitative analysis over 500 million web articles, making this the largest-scale study ever conducted on the topic.
arXiv Detail & Related papers (2020-08-17T07:13:24Z) - Object-QA: Towards High Reliable Object Quality Assessment [71.71188284059203]
In object recognition applications, object images usually appear with different quality levels.
We propose an effective approach named Object-QA to assess high-reliable quality scores for object images.
arXiv Detail & Related papers (2020-05-27T01:46:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.