Improving Random Testing via LLM-powered UI Tarpit Escaping for Mobile Apps

• URL: http://arxiv.org/abs/2604.06763v2
• Date: Mon, 13 Apr 2026 06:22:25 GMT
• Title: Improving Random Testing via LLM-powered UI Tarpit Escaping for Mobile Apps
• Abstract summary: We introduce LLM-powered random GUI Testing, a novel hybrid testing approach to mitigating UI tarpits during random testing.<n>Our approach monitors UI similarity to identify tarpits and query LLMs to suggest promising events for escaping the encountered tarpits.<n> HybridMonkey and HybridDroidbot outperform all baselines, achieving average coverage improvements of 54.8% and 44.8%, respectively.
• Score: 5.901783684504153
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Random GUI testing is a widely-used technique for testing mobile apps. However, its effectiveness is limited by the notorious issue -- UI exploration tarpits, where the exploration is trapped in local UI regions, thus impeding test coverage and bug discovery. In this experience paper, we introduce LLM-powered random GUI Testing, a novel hybrid testing approach to mitigating UI tarpits during random testing. Our approach monitors UI similarity to identify tarpits and query LLMs to suggest promising events for escaping the encountered tarpits. We implement our approach on top of two different automated input generation (AIG) tools for mobile apps: (1) HybridMonkey upon Monkey, a state-of-the-practice tool; and (2) HybridDroidbot upon Droidbot, a state-of-the-art tool. We evaluated them on 12 popular, real-world apps. The results show that HybridMonkey and HybridDroidbot outperform all baselines, achieving average coverage improvements of 54.8% and 44.8%, respectively, and detecting the highest number of unique crashes. In total, we found 75 unique bugs, including 34 previously unknown bugs. To date, 26 bugs have been confirmed and fixed. We also applied HybridMonkey on WeChat, a popular industrial app with billions of monthly active users. HybridMonkey achieved higher activity coverage and found more bugs than random testing.

Related papers

• GraphDroid: Asynchronous LLM-Based Mobile App GUI Testing via History-Aware Exploration and Hybrid Intent Fulfillment [15.320789851146543]
  GraphDroid is an intent-driven GUI testing framework that integrates a cluster-based memory mechanism.<n>We evaluate GraphDroid on 41 real-world Android apps against six state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-09-09T11:04:53Z)
• Can LLMs Test Terminal User Interfaces? [18.37827501923231]
  Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment.<n>Only 12% of test code exercises the interface, and 45% of those tests never send input.<n>We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript.
  arXiv  Detail & Related papers  (2026-08-04T14:36:44Z)
• Qwen-CUA: Native Computer Use for (almost) Everything [104.78368489343713]
  We introduce Qwen-CUA, a native computer-use agent with a mixture-of-experts backbone.<n>It observes only screenshots and acts through keyboard and mouse events, without DOM trees, accessibility metadata, or task-specific APIs.<n>Its scaffold maintains up to 20 active screenshots and folds older visual history in fixed-size blocks to retain recent evidence.
  arXiv  Detail & Related papers  (2026-08-03T15:04:20Z)
• AutoRestTest at the SBFT 2026 Tool Competition [48.72192947268706]
  AutoRestTest combines a Semantic Property Dependency Graph, multi-agent reinforcement learning, and large language models.<n>In the SBFT 2026 REST League, AutoRestTest ranked first in all three evaluation categories.
  arXiv  Detail & Related papers  (2026-07-01T15:24:38Z)
• Nexus: Execution-Grounded Multi-Agent Test Oracle Synthesis [57.40527331817245]
  Test oracle generation in non-regression testing is a longstanding challenge in software engineering.<n>We introduce Nexus, a novel multi-agent framework to address this challenge.
  arXiv  Detail & Related papers  (2025-10-30T12:20:25Z)
• VLM-Fuzz: Vision Language Model Assisted Recursive Depth-first Search Exploration for Effective UI Testing of Android Apps [6.122273281101832]
  Testing Android apps effectively requires a systematic exploration of the app's possible states.<n>We propose a novel, automated fuzzing approach called VLM-Fuzz for effective UI testing of Android apps.
  arXiv  Detail & Related papers  (2025-04-16T00:19:31Z)
• Model Equality Testing: Which Model Is This API Serving? [59.005869726179455]
  API providers may quantize, watermark, or finetune the underlying model, changing the output distribution.<n>We formalize detecting such distortions by Model Equality Testing, a two-sample testing problem.<n>A test built on a simple string kernel achieves a median of 77.4% power against a range of distortions.
  arXiv  Detail & Related papers  (2024-10-26T18:34:53Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• Practical Non-Intrusive GUI Exploration Testing with Visual-based Robotic Arms [14.3266199543725]
  We propose a practical non-intrusive GUI testing framework with visual robotic arms. RoboTest integrates novel GUI screen and widget detection algorithms, adaptive to detecting screens of different sizes. We evaluate RoboTest with 20 mobile apps, with a case study on an embedded system.
  arXiv  Detail & Related papers  (2023-12-17T09:05:39Z)
• Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisions [23.460051600514806]
  GPTDroid is a Q&A-based GUI testing framework for mobile apps. We introduce a functionality-aware memory prompting mechanism. It outperforms the best baseline by 32% in activity coverage, and detects 31% more bugs at a faster rate.
  arXiv  Detail & Related papers  (2023-10-24T12:30:26Z)
• Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model [23.460051600514806]
  This paper proposes InputBlaster to automatically generate unusual text inputs for mobile app crash detection. It formulates the unusual inputs generation problem as a task of producing a set of test generators, each of which can yield a batch of unusual text inputs. It is evaluated on 36 text input widgets with cash bugs involving 31 popular Android apps, and results show that it achieves 78% bug detection rate, with 136% higher than the best baseline.
  arXiv  Detail & Related papers  (2023-10-24T09:10:51Z)
• Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing [23.460051600514806]
  We propose GPTDroid, asking Large Language Model to chat with the mobile apps by passing the GUI page information to LLM to elicit testing scripts. Within it, we extract the static context of the GUI page and the dynamic context of the iterative testing process. We evaluate GPTDroid on 86 apps from Google Play, and its activity coverage is 71%, with 32% higher than the best baseline, and can detect 36% more bugs with faster speed than the best baseline.
  arXiv  Detail & Related papers  (2023-05-16T13:46:52Z)
• BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models [73.29106813131818]
  bias testing is currently cumbersome since the test sentences are generated from a limited set of manual templates or need expensive crowd-sourcing. We propose using ChatGPT for the controllable generation of test sentences, given any arbitrary user-specified combination of social groups and attributes. We present an open-source comprehensive bias testing framework (BiasTestGPT), hosted on HuggingFace, that can be plugged into any open-source PLM for bias testing.
  arXiv  Detail & Related papers  (2023-02-14T22:07:57Z)
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design [63.48989885374238]
  When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testing algorithms that can operate in a noisy setting.
  arXiv  Detail & Related papers  (2020-04-26T23:41:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.