A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning

• URL: http://arxiv.org/abs/2201.06044v1
• Date: Sun, 16 Jan 2022 13:13:29 GMT
• Title: A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning
• Abstract summary: Since test suites are costly to execute, industry needs methods for test case prioritisation ( TCP) Recent TCP methods use machine learning (ML) to exploit the information known about the system under test (SUT) and its test cases. This paper analyses two decades of TCP research, and presents a taxonomy of 91 information attributes that have been used.
• Score: 12.24664973838839
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Most software companies have extensive test suites and re-run parts of them continuously to ensure recent changes have no adverse effects. Since test suites are costly to execute, industry needs methods for test case prioritisation (TCP). Recently, TCP methods use machine learning (ML) to exploit the information known about the system under test (SUT) and its test cases. However, the value added by ML-based TCP methods should be critically assessed with respect to the cost of collecting the information. This paper analyses two decades of TCP research, and presents a taxonomy of 91 information attributes that have been used. The attributes are classified with respect to their information sources and the characteristics of their extraction process. Based on this taxonomy, TCP methods validated with industrial data and those applying ML are analysed in terms of information availability, attribute combination and definition of data features suitable for ML. Relying on a high number of information attributes, assuming easy access to SUT code and simplified testing environments are identified as factors that might hamper industrial applicability of ML-based TCP. The TePIA taxonomy provides a reference framework to unify terminology and evaluate alternatives considering the cost-benefit of the information attributes.

Related papers

• Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs [0.0]
  This paper investigates whether Large Language Models can support taxonomy-agnostic annotation of explicitly transmitted PII values in HTTP message bodies.<n>We introduce a multi-stage LLM-based pipeline that combines deterministic pre-processing with label-level classification, targeted instance-level value annotation, and output validation.<n>Results show that the pipeline accurately detects PII types and extracts corresponding values for concrete PII.
  arXiv  Detail & Related papers  (2026-05-07T14:08:22Z)
• ROSE: Retrieval-Oriented Segmentation Enhancement [79.7370065070812]
  Existing segmentation models based on multimodal large language models (MLLMs) often struggle with novel or emerging entities due to their inability to incorporate up-to-date knowledge.<n>We introduce the Novel Emerging Task (NEST), which focuses on segmenting (i) novel entities that MLLMs fail to recognize due to their absence from training data, and (ii) emerging entities that exist within the model's knowledge but demand up-to-date information for accurate recognition.<n>Rose: Retrieval-Oriented Enhancement, a plug-and-play framework designed to augment any MLLM-based segmentation model,
  arXiv  Detail & Related papers  (2026-04-15T17:59:35Z)
• CAPID: Context-Aware PII Detection for Question-Answering Systems [2.538582648751871]
  We propose CAPID, a practical approach that fine-tunes a locally owned small language model (SLM) that filters sensitive information before it is passed to LLMs for QA.<n>Existing datasets do not capture the context-dependent relevance of PII needed to train such a model effectively.<n>Our experiments show that relevance-aware PII detection with a fine-tuned SLM substantially outperforms existing baselines in span, relevance and type accuracy.
  arXiv  Detail & Related papers  (2026-02-10T18:41:31Z)
• Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures [0.38180404292108383]
  Existing approaches either depend on non-scalable, manual labeling, or on ML classifiers that do not properly support the diversity of the domain.<n>We evaluate whether Small Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations.
  arXiv  Detail & Related papers  (2026-01-07T15:00:22Z)
• Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systems [54.916243942641444]
  Large language models (LLMs) are emerging as key enablers of automation in domains such as telecommunications.<n>We study an edge-cloud-expert cascaded LLM-based knowledge system that supports decision-making through a question-and-answer pipeline.
  arXiv  Detail & Related papers  (2025-12-23T03:10:09Z)
• Self-Refinement Strategies for LLM-based Product Attribute Value Extraction [51.45146101802871]
  This paper investigates applying two self-refinement techniques to the product attribute value extraction task. The experiments show that both self-refinement techniques fail to significantly improve the extraction performance while substantially increasing processing costs. For scenarios with development data, fine-tuning yields the highest performance, while the ramp-up costs of fine-tuning are balanced out as the amount of product descriptions increases.
  arXiv  Detail & Related papers  (2025-01-02T12:55:27Z)
• Assessing data-driven predictions of band gap and electrical conductivity for transparent conducting materials [10.3054383984768]
  We propose a data-driven framework aimed at accelerating the discovery of new transparent conducting materials. To mitigate the shortage of available data, we create and validate unique experimental databases. We test our approach on a list of 55 compositions containing typical elements of known TCMs.
  arXiv  Detail & Related papers  (2024-11-21T11:37:05Z)
• Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models [49.06068319380296]
  We introduce context-aware testing (CAT) which uses context as an inductive bias to guide the search for meaningful model failures. We instantiate the first CAT system, SMART Testing, which employs large language models to hypothesize relevant and likely failures.
  arXiv  Detail & Related papers  (2024-10-31T15:06:16Z)
• BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping [64.8477128397529]
  We propose a training-required and training-free test-time adaptation framework. We maintain a light-weight key-value memory for feature retrieval from instance-agnostic historical samples and instance-aware boosting samples. We theoretically justify the rationality behind our method and empirically verify its effectiveness on both the out-of-distribution and the cross-domain datasets.
  arXiv  Detail & Related papers  (2024-10-20T15:58:43Z)
• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [83.90988015005934]
  Uncertainty quantification (UQ) is a critical component of machine learning (ML) applications. We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines. We conduct a large-scale empirical investigation of UQ and normalization techniques across nine tasks, and identify the most promising approaches.
  arXiv  Detail & Related papers  (2024-06-21T20:06:31Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Wildest Dreams: Reproducible Research in Privacy-preserving Neural Network Training [2.853180143237022]
  This work focuses on the ML model's training phase, where maintaining user data privacy is of utmost importance. We provide a solid theoretical background that eases the understanding of current approaches. We reproduce results for some of the papers and examine at what level existing works in the field provide support for open science.
  arXiv  Detail & Related papers  (2024-03-06T10:25:36Z)
• Prospector Heads: Generalized Feature Attribution for Large Models & Data [82.02696069543454]
  We introduce prospector heads, an efficient and interpretable alternative to explanation-based attribution methods. We demonstrate how prospector heads enable improved interpretation and discovery of class-specific patterns in input data.
  arXiv  Detail & Related papers  (2024-02-18T23:01:28Z)
• An empirical study of testing machine learning in the wild [35.13282520395855]
  Machine and deep learning (ML/DL) algorithms have been increasingly adopted in many software systems. Due to their inductive nature, ensuring the quality of these systems remains a significant challenge for the research community. Recent research in ML/DL quality assurance has adapted concepts from traditional software testing, such as mutation testing, to improve reliability.
  arXiv  Detail & Related papers  (2023-12-19T21:18:14Z)
• Leveraging Large Language Models to Improve REST API Testing [51.284096009803406]
  RESTGPT takes as input an API specification, extracts machine-interpretable rules, and generates example parameter values from natural-language descriptions in the specification. Our evaluations indicate that RESTGPT outperforms existing techniques in both rule extraction and value generation.
  arXiv  Detail & Related papers  (2023-12-01T19:53:23Z)
• ezDPS: An Efficient and Zero-Knowledge Machine Learning Inference Pipeline [2.0813318162800707]
  We propose ezDPS, a new efficient and zero-knowledge Machine Learning inference scheme. ezDPS is a zkML pipeline in which the data is processed in multiple stages for high accuracy. We show that ezDPS achieves one-to-three orders of magnitude more efficient than the generic circuit-based approach in all metrics.
  arXiv  Detail & Related papers  (2022-12-11T06:47:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.