TestLab: An Intelligent Automated Software Testing Framework
- URL: http://arxiv.org/abs/2306.03602v1
- Date: Tue, 6 Jun 2023 11:45:22 GMT
- Title: TestLab: An Intelligent Automated Software Testing Framework
- Authors: Tiago Dias, Arthur Batista, Eva Maia and Isabel Pra\c{c}a
- Abstract summary: TestLab is an automated software testing framework that attempts to gather a set of testing methods and automate them using Artificial Intelligence.
The first two modules aim to identify vulnerabilities from different perspectives, while the third module enhances traditional automated software testing by automatically generating test cases.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The prevalence of software systems has become an integral part of modern-day
living. Software usage has increased significantly, leading to its growth in
both size and complexity. Consequently, software development is becoming a more
time-consuming process. In an attempt to accelerate the development cycle, the
testing phase is often neglected, leading to the deployment of flawed systems
that can have significant implications on the users daily activities. This work
presents TestLab, an intelligent automated software testing framework that
attempts to gather a set of testing methods and automate them using Artificial
Intelligence to allow continuous testing of software systems at multiple levels
from different scopes, ranging from developers to end-users. The tool consists
of three modules, each serving a distinct purpose. The first two modules aim to
identify vulnerabilities from different perspectives, while the third module
enhances traditional automated software testing by automatically generating
test cases through source code analysis.
Related papers
- Harnessing the Power of LLMs: Automating Unit Test Generation for High-Performance Computing [7.3166218350585135]
Unit testing is crucial in software engineering for ensuring quality.
It's not widely used in parallel and high-performance computing software, particularly scientific applications.
We propose an automated method for generating unit tests for such software.
arXiv Detail & Related papers (2024-07-06T22:45:55Z) - A Comprehensive Study on Automated Testing with the Software Lifecycle [0.6144680854063939]
The research examines how automated testing makes it easier to evaluate software quality, how it saves time as compared to manual testing, and how it differs from each of them in terms of benefits and drawbacks.
The process of testing software applications is simplified, customized to certain testing situations, and can be successfully carried out by using automated testing tools.
arXiv Detail & Related papers (2024-05-02T06:30:37Z) - DevBench: A Comprehensive Benchmark for Software Development [72.24266814625685]
DevBench is a benchmark that evaluates large language models (LLMs) across various stages of the software development lifecycle.
Empirical studies show that current LLMs, including GPT-4-Turbo, fail to solve the challenges presented within DevBench.
Our findings offer actionable insights for the future development of LLMs toward real-world programming applications.
arXiv Detail & Related papers (2024-03-13T15:13:44Z) - MultiTest: Physical-Aware Object Insertion for Testing Multi-sensor
Fusion Perception Systems [23.460181958075566]
Multi-sensor fusion (MSF) is a key technique in addressing numerous safety-critical tasks and applications, e.g., self-driving cars and automated robotic arms.
Existing testing methods primarily concentrate on single-sensor perception systems.
We introduce MultiTest, a fitness-guided metamorphic testing method for complex MSF perception systems.
arXiv Detail & Related papers (2024-01-25T17:03:02Z) - Constraint-Guided Test Execution Scheduling: An Experience Report at ABB
Robotics [13.50507740574158]
We present the results of a project called DynTest whose goal is to automate the scheduling of test execution from a large test repository.
This paper reports on our experience and lessons learned for successfully transferring constraint-based optimization models for test execution scheduling at ABB Robotics.
arXiv Detail & Related papers (2023-06-02T13:29:32Z) - ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data
Format [88.33443450434521]
Task-oriented dialogue (TOD) systems function as digital assistants, guiding users through various tasks such as booking flights or finding restaurants.
Existing toolkits for building TOD systems often fall short of in delivering comprehensive arrays of data, models, and experimental environments.
We introduce ConvLab-3: a multifaceted dialogue system toolkit crafted to bridge this gap.
arXiv Detail & Related papers (2022-11-30T16:37:42Z) - MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation [104.48766162008815]
We propose and explore a new multi-modal extension of test-time adaptation for 3D semantic segmentation.
To design a framework that can take full advantage of multi-modality, each modality provides regularized self-supervisory signals to other modalities.
Our regularized pseudo labels produce stable self-learning signals in numerous multi-modal test-time adaptation scenarios.
arXiv Detail & Related papers (2022-04-27T02:28:12Z) - SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video
Games Using Risk Based Testing and Machine Learning [62.997667081978825]
Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems.
We present SUPERNOVA, a system responsible for test selection and defect prevention while also functioning as an automation hub.
The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title.
arXiv Detail & Related papers (2022-03-10T00:47:46Z) - Automated User Experience Testing through Multi-Dimensional Performance
Impact Analysis [0.0]
We propose a novel automated user experience testing methodology.
It learns how code changes impact the time unit and system tests take, and extrapolates user experience changes based on this information.
Our open-source tool achieved 3.7% mean absolute error rate with a random forest regressor.
arXiv Detail & Related papers (2021-04-08T01:18:01Z) - Integrated Benchmarking and Design for Reproducible and Accessible
Evaluation of Robotic Agents [61.36681529571202]
We describe a new concept for reproducible robotics research that integrates development and benchmarking.
One of the central components of this setup is the Duckietown Autolab, a standardized setup that is itself relatively low-cost and reproducible.
We validate the system by analyzing the repeatability of experiments conducted using the infrastructure and show that there is low variance across different robot hardware and across different remote labs.
arXiv Detail & Related papers (2020-09-09T15:31:29Z) - Beyond Accuracy: Behavioral Testing of NLP models with CheckList [66.42971817954806]
CheckList is a task-agnostic methodology for testing NLP models.
CheckList includes a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation.
In a user study, NLP practitioners with CheckList created twice as many tests, and found almost three times as many bugs as users without it.
arXiv Detail & Related papers (2020-05-08T15:48:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.