A Feature-Based Approach to Generating Comprehensive End-to-End Tests

• URL: http://arxiv.org/abs/2408.01894v1
• Date: Sun, 4 Aug 2024 01:16:04 GMT
• Title: A Feature-Based Approach to Generating Comprehensive End-to-End Tests
• Authors: Parsa Alian, Noor Nashid, Mobina Shahbandeh, Taha Shabani, Ali Mesbah,
• Abstract summary: AUTOE2E is a novel approach to automate the generation of semantically meaningful feature-driven E2E test cases for web applications. We introduce E2EBENCH, a new benchmark for automatically assessing the feature coverage of E2E test suites.
• Score: 5.7340627516257525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end (E2E) testing is essential for ensuring web application quality. However, manual test creation is time-consuming and current test generation techniques produce random tests. In this paper, we present AUTOE2E, a novel approach that leverages Large Language Models (LLMs) to automate the generation of semantically meaningful feature-driven E2E test cases for web applications. AUTOE2E intelligently infers potential features within a web application and translates them into executable test scenarios. Furthermore, we address a critical gap in the research community by introducing E2EBENCH, a new benchmark for automatically assessing the feature coverage of E2E test suites. Our evaluation on E2EBENCH demonstrates that AUTOE2E achieves an average feature coverage of 79%, outperforming the best baseline by 558%, highlighting its effectiveness in generating high-quality, comprehensive test cases.

Related papers

• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• AI-powered test automation tools: A systematic review and empirical evaluation [1.3490988186255937]
  We investigate the features provided by existing AI-based test automation tools. We empirically evaluate how the AI features can be helpful for effectiveness and efficiency of testing. We also study the limitations of the AI features in AI-based test tools.
  arXiv  Detail & Related papers  (2024-08-31T10:10:45Z)
• A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites [1.4563527353943984]
  Large Language Models (LLMs) have been applied to various aspects of software development. We present AgoneTest: an automated system for generating test suites for Java projects.
  arXiv  Detail & Related papers  (2024-08-14T23:02:16Z)
• Selene: Pioneering Automated Proof in Software Verification [62.09555413263788]
  We introduce Selene, which is the first project-level automated proof benchmark constructed based on the real-world industrial-level operating system microkernel, seL4. Our experimental results with advanced large language models (LLMs), such as GPT-3.5-turbo and GPT-4, highlight the capabilities of LLMs in the domain of automated proof generation.
  arXiv  Detail & Related papers  (2024-01-15T13:08:38Z)
• End-to-End Test Coverage Metrics in Microservice Systems: An Automated Approach [2.6245844272542027]
  This paper introduces test coverage metrics for evaluating the extent of E2E test suite coverage for microservice endpoints. Next, it presents an automated approach to compute these metrics to provide feedback on the completeness of E2E test suites.
  arXiv  Detail & Related papers  (2023-08-18T02:30:19Z)
• From Static Benchmarks to Adaptive Testing: Psychometrics in AI Evaluation [60.14902811624433]
  We discuss a paradigm shift from static evaluation methods to adaptive testing. This involves estimating the characteristics and value of each test item in the benchmark and dynamically adjusting items in real-time. We analyze the current approaches, advantages, and underlying reasons for adopting psychometrics in AI evaluation.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Neural Embeddings for Web Testing [49.66745368789056]
  Existing crawlers rely on app-specific, threshold-based, algorithms to assess state equivalence. We propose WEBEMBED, a novel abstraction function based on neural network embeddings and threshold-free classifiers. Our evaluation on nine web apps shows that WEBEMBED outperforms state-of-the-art techniques by detecting near-duplicates more accurately.
  arXiv  Detail & Related papers  (2023-06-12T19:59:36Z)
• E-Valuating Classifier Two-Sample Tests [11.248868528186332]
  Our test combines ideas from existing work-valid split likelihood ratio tests and predictive independence tests. The resulting E-values are suitable for anytime sequential two-sample tests.
  arXiv  Detail & Related papers  (2022-10-24T08:18:36Z)
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition [65.84978547406753]
  Test-time Adaptation aims to adapt the model trained on source domains to yield better predictions for test samples. Single-Utterance Test-time Adaptation (SUTA) is the first TTA study in speech area to our best knowledge.
  arXiv  Detail & Related papers  (2022-03-27T06:38:39Z)
• Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification [59.698811329287174]
  We leverage GPT-2 for generating artificial training instances in order to improve classification performance. Our results show that fine-tuning GPT-2 in a handful of label instances leads to consistent classification improvements.
  arXiv  Detail & Related papers  (2021-11-17T12:10:03Z)
• On Introducing Automatic Test Case Generation in Practice: A Success Story and Lessons Learned [7.717446055777458]
  This paper reports our experience in introducing techniques for automatically generating system test suites in a medium-size company. We describe the technical and organisational obstacles that we faced when introducing automatic test case generation. We present ABT2.0, the test case generator that we developed.
  arXiv  Detail & Related papers  (2021-02-28T11:31:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.