Test Script Intention Generation for Mobile Application via GUI Image and Code Understanding

• URL: http://arxiv.org/abs/2107.05165v3
• Date: Mon, 03 Mar 2025 13:37:56 GMT
• Title: Test Script Intention Generation for Mobile Application via GUI Image and Code Understanding
• Authors: Shengcheng Yu, Chunrong Fang, Jia Liu, Zhenyu Chen,
• Abstract summary: Test scripts play a more important role in mobile app testing than test cases for source code.<n>TestIntention is a novel approach to infer the intention of GUI test scripts.<n>Results of all operations are combined to generate test intention for test scripts.
• Score: 12.973016336177047
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Testing is the most direct and effective technique to ensure software quality. Test scripts always play a more important role in mobile app testing than test cases for source code, due to the GUI-intensive and event-driven characteristics of mobile applications (app). Test scripts focus on user interactions and the corresponding response events, which is significant for testing the target app functionalities. Therefore, it is critical to understand the test scripts for better script maintenance and modification. There exist some mature code understanding (i.e., code comment generation) technologies that can be directly applied to functionality source code with business logic. However, such technologies will have difficulties when being applied to test scripts, because test scripts are loosely linked to apps under test (AUT) by widget selectors, and do not contain business logic themselves. In order to solve the test script understanding gap, this paper presents a novel approach, namely TestIntention, to infer the intention of GUI test scripts. Test intention refers to the user expectations of app behaviors for specific operations. TestIntention formalizes test scripts with an operation sequence model. For each operation within the sequence, TestIntention extracts the target widget selector and links the selector to the GUI layout information or the corresponding response events. For widgets identified by XPath, TestIntention utilizes the image understanding technologies to explore the detailed information of the widget images, the intention of which is understood with a deep learning model. For widgets identified by ID, TestIntention first maps the selectors to the response methods with business logic, and then adopts code understanding technologies to describe code in natural language form. Results of all operations are combined to generate test intention for test scripts.

Related papers

• Skill Discovery for Software Scripting Automation via Offline Simulations with LLMs [63.10710876536337]
  We propose an offline simulation framework to curate a software-specific skillset, a collection of verified scripts. Our framework comprises two components: (1) task creation, using top-down functionality and bottom-up API synergy exploration to generate helpful tasks. Experiments with Adobe Illustrator demonstrate that our framework significantly improves automation success rates, reduces response time, and saves runtime token costs.
  arXiv  Detail & Related papers  (2025-04-29T04:03:37Z)
• CLOVER: A Test Case Generation Benchmark with Coverage, Long-Context, and Verification [71.34070740261072]
  This paper presents a benchmark, CLOVER, to evaluate models' capabilities in generating and completing test cases.<n>The benchmark is containerized for code execution across tasks, and we will release the code, data, and construction methodologies.
  arXiv  Detail & Related papers  (2025-02-12T21:42:56Z)
• AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL [46.65963514391019]
  AutoRestTest is a novel tool that integrates the Semantic Property Dependency Graph (SPDG) with Multi-Agent Reinforcement Learning (MARL) and large language models (LLMs) for effective REST API testing.
  arXiv  Detail & Related papers  (2025-01-15T05:54:33Z)
• LlamaRestTest: Effective REST API Testing with Small Language Models [50.058600784556816]
  We present LlamaRestTest, a novel approach that employs two custom LLMs to generate realistic test inputs.<n>LlamaRestTest surpasses state-of-the-art tools in code coverage and error detection, even with RESTGPT-enhanced specifications.
  arXiv  Detail & Related papers  (2025-01-15T05:51:20Z)
• GUI Testing Arena: A Unified Benchmark for Advancing Autonomous GUI Testing Agent [24.97846085313314]
  We propose a formalized and comprehensive environment to evaluate the entire process of automated GUI Testing.<n>We divide the testing process into three key subtasks: test intention generation, test task execution, and GUI defect detection.<n>It evaluates the performance of different models using three data types: real mobile applications, mobile applications with artificially injected defects, and synthetic data.
  arXiv  Detail & Related papers  (2024-12-24T13:41:47Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch.<n>Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests.<n> Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)
• AUITestAgent: Automatic Requirements Oriented GUI Function Testing [12.83932274541321]
  This paper introduces AUITestAgent, the first automatic, natural language-driven GUI testing tool for mobile apps. It is capable of fully automating the entire process of GUI interaction and function verification. Experiments on customized benchmarks demonstrate that AUITestAgent outperforms existing tools in the quality of generated GUI interactions.
  arXiv  Detail & Related papers  (2024-07-12T06:14:46Z)
• Practical, Automated Scenario-based Mobile App Testing [13.52057950260007]
  Test scripts developed by human testers consider business logic by focusing on testing scenarios. Due to the GUI-intensive feature of mobile apps, human testers always understand app GUI to organize test scripts for scenarios. ScenTest tries to start automated testing by imitating human practices and integrating domain knowledge into scenario-based mobile app testing.
  arXiv  Detail & Related papers  (2024-06-12T15:48:39Z)
• Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing [17.24045904273874]
  We propose DroidAgent, an autonomous GUI testing agent for Android. It is based on Large Language Models and support mechanisms such as long- and short-term memory. DroidAgent achieved 61% activity coverage, compared to 51% for current state-of-the-art GUI testing techniques.
  arXiv  Detail & Related papers  (2023-11-15T01:59:40Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing [23.460051600514806]
  We propose GPTDroid, asking Large Language Model to chat with the mobile apps by passing the GUI page information to LLM to elicit testing scripts. Within it, we extract the static context of the GUI page and the dynamic context of the iterative testing process. We evaluate GPTDroid on 86 apps from Google Play, and its activity coverage is 71%, with 32% higher than the best baseline, and can detect 36% more bugs with faster speed than the best baseline.
  arXiv  Detail & Related papers  (2023-05-16T13:46:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.