A Multi-Year Grey Literature Review on AI-assisted Test Automation

• URL: http://arxiv.org/abs/2408.06224v1
• Date: Mon, 12 Aug 2024 15:26:36 GMT
• Title: A Multi-Year Grey Literature Review on AI-assisted Test Automation
• Authors: Filippo Ricca, Alessandro Marchetto, Andrea Stocco,
• Abstract summary: Test Automation (TA) techniques are crucial for quality assurance in software engineering. TA techniques face limitations such as high test suite maintenance costs and the need for extensive programming skills. Artificial Intelligence (AI) offers new opportunities to address these issues through automation and improved practices.
• Score: 46.97326049485643
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context: Test Automation (TA) techniques are crucial for quality assurance in software engineering but face limitations such as high test suite maintenance costs and the need for extensive programming skills. Artificial Intelligence (AI) offers new opportunities to address these issues through automation and improved practices. Objectives: This study surveys grey literature to explore how AI is adopted in TA, focusing on the problems it solves, its solutions, and the available tools. Additionally, the study gathers expert insights to understand AI's current and future role in TA. Methods: We reviewed over 3,600 grey literature sources over five years, including blogs, white papers, and user manuals, and finally filtered 342 documents to develop taxonomies of TA problems and AI solutions. We also cataloged 100 AI-driven TA tools and interviewed five expert software testers to gain insights into AI's current and future role in TA. Results: The study found that manual test code development and maintenance are the main challenges in TA. In contrast, automated test generation and self-healing test scripts are the most common AI solutions. We identified 100 AI-based TA tools, with Applitools, Testim, Functionize, AccelQ, and Mabl being the most adopted in practice. Conclusion: This paper offers a detailed overview of AI's impact on TA through grey literature analysis and expert interviews. It presents new taxonomies of TA problems and AI solutions, provides a catalog of AI-driven tools, and relates solutions to problems and tools to solutions. Interview insights further revealed the state and future potential of AI in TA. Our findings support practitioners in selecting TA tools and guide future research directions.

Related papers

• The Role of Artificial Intelligence and Machine Learning in Software Testing [0.14896196009851972]
  Artificial Intelligence (AI) and Machine Learning (ML) have significantly impacted various industries. Software testing, a crucial part of the software development lifecycle (SDLC), ensures the quality and reliability of software products. This paper explores the role of AI and ML in software testing by reviewing existing literature, analyzing current tools and techniques, and presenting case studies.
  arXiv  Detail & Related papers  (2024-09-04T13:25:13Z)
• AI-powered test automation tools: A systematic review and empirical evaluation [1.3490988186255937]
  We investigate the features provided by existing AI-based test automation tools. We empirically evaluate how the AI features can be helpful for effectiveness and efficiency of testing. We also study the limitations of the AI features in AI-based test tools.
  arXiv  Detail & Related papers  (2024-08-31T10:10:45Z)
• AI-Tutoring in Software Engineering Education [0.7631288333466648]
  We conducted an exploratory case study by integrating the GPT-3.5-Turbo model as an AI-Tutor within the APAS Artemis. The findings highlight advantages, such as timely feedback and scalability. However, challenges like generic responses and students' concerns about a learning progress inhibition when using the AI-Tutor were also evident.
  arXiv  Detail & Related papers  (2024-04-03T08:15:08Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Learning from Teaching Assistants to Program with Subgoals: Exploring the Potential for AI Teaching Assistants [18.14390906820148]
  We investigate the practicality of using generative AI as TAs in programming education by examining novice learners' interaction with TAs in a subgoal learning environment. Our study shows that learners can solve tasks faster with comparable scores with AI TAs. We suggest guidelines to better design and utilize generative AI as TAs in programming education from the result of our chat log analysis.
  arXiv  Detail & Related papers  (2023-09-19T08:30:58Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• AI Explainability 360: Impact and Design [120.95633114160688]
  In 2019, we created AI Explainability 360 (Arya et al. 2020), an open source software toolkit featuring ten diverse and state-of-the-art explainability methods. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.
  arXiv  Detail & Related papers  (2021-09-24T19:17:09Z)
• Measuring Ethics in AI with AI: A Methodology and Dataset Construction [1.6861004263551447]
  We propose to use such newfound capabilities of AI technologies to augment our AI measuring capabilities. We do so by training a model to classify publications related to ethical issues and concerns. We highlight the implications of AI metrics, in particular their contribution towards developing trustful and fair AI-based tools and technologies.
  arXiv  Detail & Related papers  (2021-07-26T00:26:12Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.