Seeding Contradiction: a fast method for generating full-coverage test suites

• URL: http://arxiv.org/abs/2309.04232v1
• Date: Fri, 8 Sep 2023 09:37:11 GMT
• Title: Seeding Contradiction: a fast method for generating full-coverage test suites
• Authors: Li Huang, Bertrand Meyer, Manuel Oriol
• Abstract summary: A test suite is a key resource for managing program evolution. Seeding Contradiction inserts incorrect instructions into every basic block of the program. The method is static, works fast, and achieves excellent coverage.
• Score: 36.52277143058971
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The regression test suite, a key resource for managing program evolution, needs to achieve 100% coverage, or very close, to be useful. Devising a test suite manually is unacceptably tedious, but existing automated methods are often inefficient. The method described in this article, ``Seeding Contradiction'', inserts incorrect instructions into every basic block of the program, enabling an SMT-based Hoare-style prover to generate a counterexample for every branch of the program and, from the collection of all such counterexamples, a test suite. The method is static, works fast, and achieves excellent coverage.

Related papers

• Efficient Dynamic Test Case Generation for Path-Based Coverage Criteria [2.099922236065961]
  We present a novel approach to test-case generation that satisfies four white-box, path-based coverage criteria.<n>Our method builds on a modified version of Johnson algorithm and enables test cases to be generated incrementally and on demand.
  arXiv  Detail & Related papers  (2026-02-21T09:26:23Z)
• Alignment with Fill-In-the-Middle for Enhancing Code Generation [56.791415642365415]
  We propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases.<n>Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench.
  arXiv  Detail & Related papers  (2025-08-27T03:15:53Z)
• iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocols [11.749977502129898]
  Large Language Models (LLMs) have demonstrated impressive text comprehension and code generation abilities.<n>We propose iPanda, the first framework that leverages LLMs to automate protocol conformance testing.<n>Experiments on various protocols show that iPanda significantly outperforms pure LLM-based approaches.
  arXiv  Detail & Related papers  (2025-07-01T02:27:44Z)
• Automated Test Generation from Program Documentation Encoded in Code Comments [4.696083734269232]
  This paper introduces a novel test generation technique that exploits the code-comment documentation constructively. We deliver test cases with names and oracles properly contextualized on the target behaviors.
  arXiv  Detail & Related papers  (2025-04-29T20:23:56Z)
• Bisimulation Learning [55.859538562698496]
  We compute finite bisimulations of state transition systems with large, possibly infinite state space. Our technique yields faster verification results than alternative state-of-the-art tools in practice.
  arXiv  Detail & Related papers  (2024-05-24T17:11:27Z)
• Large Language Models to Generate System-Level Test Programs Targeting Non-functional Properties [3.3305233186101226]
  This paper proposes Large Language Models (LLMs) to generate test programs. We take a first glance at how pre-trained LLMs perform in test program generation to optimize non-functional properties of the DUT.
  arXiv  Detail & Related papers  (2024-03-15T08:01:02Z)
• Advancing BDD Software Testing: Dynamic Scenario Re-Usability And Step Auto-Complete For Cucumber Framework [0.0]
  This paper presents and implements the re-usability of scenarios within scenarios for behavior-driven development (BDD) Gherkin test scripts in the Cucumber Java framework. The paper also dives a little into the limitations of Cucumber single-threaded scenario execution model.
  arXiv  Detail & Related papers  (2024-02-24T23:15:39Z)
• Code-Aware Prompting: A study of Coverage Guided Test Generation in Regression Setting using LLM [32.44432906540792]
  We present SymPrompt, a code-aware prompting strategy for large language models in test generation. SymPrompt enhances correct test generations by a factor of 5 and bolsters relative coverage by 26% for CodeGen2. Notably, when applied to GPT-4, SymPrompt improves coverage by over 2x compared to baseline prompting strategies.
  arXiv  Detail & Related papers  (2024-01-31T18:21:49Z)
• Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization [64.62570402941387]
  We use a single test sample to adapt multi-modal prompts at test time by minimizing the feature distribution shift to bridge the gap in the test domain. Our method improves zero-shot top- 1 accuracy beyond existing prompt-learning techniques, with a 3.08% improvement over the baseline MaPLe.
  arXiv  Detail & Related papers  (2023-11-02T17:59:32Z)
• Understanding prompt engineering may not require rethinking generalization [56.38207873589642]
  We show that the discrete nature of prompts, combined with a PAC-Bayes prior given by a language model, results in generalization bounds that are remarkably tight by the standards of the literature. This work provides a possible justification for the widespread practice of prompt engineering.
  arXiv  Detail & Related papers  (2023-10-06T00:52:48Z)
• Test Case Recommendations with Distributed Representation of Code Syntactic Features [2.225268436173329]
  We propose an automated approach which exploits both structural and semantic properties of source code methods and test cases. The proposed approach initially trains a neural network to transform method-level source code, as well as unit tests, into distributed representations. The model computes cosine similarity between the method's embedding and the previously-embedded training instances.
  arXiv  Detail & Related papers  (2023-10-04T21:42:01Z)
• Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification [0.0]
  We introduce a novel method for automatic constraint extraction and test selection. Coverage-directed test selection is based on supervised learning from coverage feedback. We show how coverage-directed test selection can reduce manual constraint writing, prioritise effective tests, reduce verification resource consumption, and accelerate coverage closure on a large, real-life industrial hardware design.
  arXiv  Detail & Related papers  (2022-05-17T17:49:30Z)
• Coping with Label Shift via Distributionally Robust Optimisation [72.80971421083937]
  We propose a model that minimises an objective based on distributionally robust optimisation (DRO) We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective.
  arXiv  Detail & Related papers  (2020-10-23T08:33:04Z)
• Provably Consistent Partial-Label Learning [120.4734093544867]
  Partial-label learning (PLL) is a multi-class classification problem, where each training example is associated with a set of candidate labels. In this paper, we propose the first generation model of candidate label sets, and develop two novel methods that are guaranteed to be consistent. Experiments on benchmark and real-world datasets validate the effectiveness of the proposed generation model and two methods.
  arXiv  Detail & Related papers  (2020-07-17T12:19:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.