Temac: Multi-Agent Collaboration for Automated Web GUI Testing

• URL: http://arxiv.org/abs/2506.00520v1
• Date: Sat, 31 May 2025 11:43:37 GMT
• Title: Temac: Multi-Agent Collaboration for Automated Web GUI Testing
• Authors: Chenxu Liu, Zhiyu Gu, Guoquan Wu, Ying Zhang, Jun Wei, Tao Xie,
• Abstract summary: We propose Temac, an approach that enhances automated web GUI testing (AWGT) using large language models (LLMs) to increase code coverage.<n>Our evaluation results show that Temac exceeds state-of-the-art approaches from 12.5% to 60.3% on average code coverage on six complex open-source web applications.
• Score: 10.661373474430604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quality assurance of web applications is critical, as web applications play an essential role in people's daily lives. To reduce labor costs, automated web GUI testing (AWGT) is widely adopted, exploring web applications via GUI actions such as clicks and text inputs. However, these approaches face limitations in generating continuous and meaningful action sequences capable of covering complex functionalities. Recent work incorporates large language models (LLMs) for GUI testing. However, these approaches face various challenges, including low efficiency of LLMs, high complexity of rich web application contexts, and a low success rate of LLMs in executing GUI tasks. To address these challenges, in this paper, we propose Temac, an approach that enhances AWGT using LLM-based multi-agent collaboration to increase code coverage. Temac is motivated by our insight that LLMs can enhance AWGT in executing complex functionalities, while the information discovered during AWGT can, in turn, be provided as the domain knowledge to improve the LLM-based task execution. Specifically, given a web application, Temac initially runs an existing approach to broadly explore application states. When the testing coverage stagnates, Temac then employs LLM-based agents to summarize the collected information to form a knowledge base and to infer not-covered functionalities. Guided by this knowledge base, Temac finally uses specialized LLM-based agents to target and execute the not-covered functionalities, reaching deeper states beyond those explored by the existing approach. Our evaluation results show that Temac exceeds state-of-the-art approaches from 12.5% to 60.3% on average code coverage on six complex open-source web applications, while revealing 445 unique failures in the top 20 real-world web applications. These results strongly demonstrate the effectiveness and the general applicability of Temac.

Related papers

• WebWalker: Benchmarking LLMs in Web Traversal [64.48425443951749]
  We introduce WebWalkerQA, a benchmark designed to assess the ability of LLMs to perform web traversal.<n>We propose WebWalker, which is a multi-agent framework that mimics human-like web navigation through an explore-critic paradigm.
  arXiv  Detail & Related papers  (2025-01-13T18:58:07Z)
• LITA: An Efficient LLM-assisted Iterative Topic Augmentation Framework [0.0]
  Large language models (LLMs) offer potential for dynamic topic refinement and discovery, yet their application often incurs high API costs.<n>To address these challenges, we propose the LLM-assisted Iterative Topic Augmentation framework (LITA)<n>LITA integrates user-provided seeds with embedding-based clustering and iterative refinement.
  arXiv  Detail & Related papers  (2024-12-17T01:43:44Z)
• AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents [52.13695464678006]
  This study enhances an LLM-based web agent by simply refining its observation and action space.<n>AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively.
  arXiv  Detail & Related papers  (2024-10-17T17:50:38Z)
• Leveraging Large Vision Language Model For Better Automatic Web GUI Testing [7.480576630392405]
  This paper proposes VETL, the first LVLM-driven endtoend web testing technique. With LVLM's scene understanding capabilities, VETL can generate valid and meaningful text inputs focusing on the local context. The selection of associated GUI elements is formulated as a visual question-answering problem, allowing LVLM to capture the logical connection between the input box and the relevant element.
  arXiv  Detail & Related papers  (2024-10-16T01:37:58Z)
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
  arXiv  Detail & Related papers  (2024-06-19T00:28:58Z)
• VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding? [115.60866817774641]
  Multimodal Large Language models (MLLMs) have shown promise in web-related tasks. evaluating their performance in the web domain remains a challenge due to the lack of comprehensive benchmarks. bench is a multimodal benchmark designed to assess the capabilities of MLLMs across a variety of web tasks.
  arXiv  Detail & Related papers  (2024-04-09T02:29:39Z)
• Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub [79.31134731122462]
  We introduce OpenAct benchmark to evaluate the open-domain task-solving capability, built on human expert consultation and repositories in GitHub.<n>We present OpenAgent, a novel LLM-based agent system that can tackle evolving queries in open domains through autonomously integrating specialized tools from GitHub.
  arXiv  Detail & Related papers  (2023-12-28T15:47:30Z)
• "What's important here?": Opportunities and Challenges of Using LLMs in Retrieving Information from Web Interfaces [19.656406003275713]
  We study how large language models (LLMs) can be used to retrieve and locate important elements for a user given query in a web interface. Our empirical experiments show that while LLMs exhibit a reasonable level of performance in retrieving important UI elements, there is still a substantial room for improvement.
  arXiv  Detail & Related papers  (2023-12-11T06:26:38Z)
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback [127.75419038610455]
  Large language models (LLMs) are able to generate human-like, fluent responses for many downstream tasks. This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules.
  arXiv  Detail & Related papers  (2023-02-24T18:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.