Constraint-Guided Test Execution Scheduling: An Experience Report at ABB
Robotics
- URL: http://arxiv.org/abs/2306.01529v1
- Date: Fri, 2 Jun 2023 13:29:32 GMT
- Title: Constraint-Guided Test Execution Scheduling: An Experience Report at ABB
Robotics
- Authors: Arnaud Gotlieb, Morten Mossige, Helge Spieker
- Abstract summary: We present the results of a project called DynTest whose goal is to automate the scheduling of test execution from a large test repository.
This paper reports on our experience and lessons learned for successfully transferring constraint-based optimization models for test execution scheduling at ABB Robotics.
- Score: 13.50507740574158
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Automated test execution scheduling is crucial in modern software development
environments, where components are frequently updated with changes that impact
their integration with hardware systems. Building test schedules, which focus
on the right tests and make optimal use of the available resources, both time
and hardware, under consideration of vast requirements on the selection of test
cases and their assignment to certain test execution machines, is a complex
optimization task. Manual solutions are time-consuming and often error-prone.
Furthermore, when software and hardware components and test scripts are
frequently added, removed or updated, static test execution scheduling is no
longer feasible and the motivation for automation taking care of dynamic
changes grows. Since 2012, our work has focused on transferring technology
based on constraint programming for automating the testing of industrial
robotic systems at ABB Robotics. After having successfully transferred
constraint satisfaction models dedicated to test case generation, we present
the results of a project called DynTest whose goal is to automate the
scheduling of test execution from a large test repository, on distinct
industrial robots. This paper reports on our experience and lessons learned for
successfully transferring constraint-based optimization models for test
execution scheduling at ABB Robotics. Our experience underlines the benefits of
a close collaboration between industry and academia for both parties.
Related papers
- Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? [73.81908518992161]
We introduce Spider2-V, the first multimodal agent benchmark focusing on professional data science and engineering.
Spider2-V features real-world tasks in authentic computer environments and incorporating 20 enterprise-level professional applications.
These tasks evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems.
arXiv Detail & Related papers (2024-07-15T17:54:37Z) - Test Oracle Automation in the era of LLMs [52.69509240442899]
Large Language Models (LLMs) have demonstrated remarkable proficiency in tackling diverse software testing tasks.
This paper aims to enable discussions on the potential of using LLMs for test oracle automation, along with the challenges that may emerge during the generation of various types of oracles.
arXiv Detail & Related papers (2024-05-21T13:19:10Z) - SERL: A Software Suite for Sample-Efficient Robotic Reinforcement
Learning [85.21378553454672]
We develop a library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment.
We find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation.
These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent robustness recovery and correction behaviors.
arXiv Detail & Related papers (2024-01-29T10:01:10Z) - AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning [54.47116888545878]
AutoAct is an automatic agent learning framework for QA.
It does not rely on large-scale annotated data and synthetic planning trajectories from closed-source models.
arXiv Detail & Related papers (2024-01-10T16:57:24Z) - TestLab: An Intelligent Automated Software Testing Framework [0.0]
TestLab is an automated software testing framework that attempts to gather a set of testing methods and automate them using Artificial Intelligence.
The first two modules aim to identify vulnerabilities from different perspectives, while the third module enhances traditional automated software testing by automatically generating test cases.
arXiv Detail & Related papers (2023-06-06T11:45:22Z) - SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video
Games Using Risk Based Testing and Machine Learning [62.997667081978825]
Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems.
We present SUPERNOVA, a system responsible for test selection and defect prevention while also functioning as an automation hub.
The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title.
arXiv Detail & Related papers (2022-03-10T00:47:46Z) - Nirikshak: A Clustering Based Autonomous API Testing Framework [0.0]
Nirikshak is a self-reliant testing framework for REST API testing.
It achieves level 2 of autonomy in executing REST API testing procedures.
Nirikshak is publicly available as an open-source software for the community at https://github.com/yashmahalwal/nirikshak.
arXiv Detail & Related papers (2021-12-15T18:05:27Z) - On Introducing Automatic Test Case Generation in Practice: A Success
Story and Lessons Learned [7.717446055777458]
This paper reports our experience in introducing techniques for automatically generating system test suites in a medium-size company.
We describe the technical and organisational obstacles that we faced when introducing automatic test case generation.
We present ABT2.0, the test case generator that we developed.
arXiv Detail & Related papers (2021-02-28T11:31:50Z) - Data Driven Testing of Cyber Physical Systems [12.93632948681342]
We propose an approach to automatically generate fault-revealing test cases for CPS.
Data collected from an application managing a smart building have been used to learn models of the environment.
arXiv Detail & Related papers (2021-02-23T04:55:10Z) - Reinforcement Learning for Test Case Prioritization [0.24366811507669126]
This paper extends recent studies on applying Reinforcement Learning to optimize testing strategies.
We test its ability to adapt to new environments, by testing it on novel data extracted from a financial institution.
We also studied the impact of using Decision Tree (DT) Approximator as a model for memory representation.
arXiv Detail & Related papers (2020-12-18T11:08:20Z) - Integrated Benchmarking and Design for Reproducible and Accessible
Evaluation of Robotic Agents [61.36681529571202]
We describe a new concept for reproducible robotics research that integrates development and benchmarking.
One of the central components of this setup is the Duckietown Autolab, a standardized setup that is itself relatively low-cost and reproducible.
We validate the system by analyzing the repeatability of experiments conducted using the infrastructure and show that there is low variance across different robot hardware and across different remote labs.
arXiv Detail & Related papers (2020-09-09T15:31:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.