From Exploration to Specification: LLM-Based Property Generation for Mobile App Testing

• URL: http://arxiv.org/abs/2604.13463v1
• Date: Wed, 15 Apr 2026 04:33:53 GMT
• Title: From Exploration to Specification: LLM-Based Property Generation for Mobile App Testing
• Abstract summary: PropGen is an automated approach for generating properties for Android apps.<n>PropGen identifies and executes valid app functionalities, generate valid properties, and repair most imprecise ones.<n>We found 25 previously unknown functional bugs in the latest versions of the subject apps.
• Score: 15.240612891177884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mobile apps often suffer from functional bugs that do not cause crashes but instead manifest as incorrect behaviors under specific user interactions. Such bugs are difficult to detect automatically because they often lack explicit test oracles. Property-based testing can effectively expose them by checking intended behavioral properties under diverse interactions. However, its use largely depends on manually written properties, whose construction is difficult and expensive, limiting its practical use for mobile apps. To address this limitation, we propose PropGen, an automated approach for generating properties for Android apps. However, this task is challenging for two reasons: app functionalities are often hard to systematically uncover and execute, and properties are difficult to derive accurately from observed behaviors. To this end, PropGen performs functionality-guided exploration to collect behavioral evidence from app executions, synthesizes properties from the collected evidence, and refines imprecise properties based on testing feedback. We implemented PropGen and evaluated it on 12 real-world Android apps. The results show that PropGen can effectively identify and execute valid app functionalities, generate valid properties, and repair most imprecise ones. Across all apps, PropGen identified 1,210 valid functionalities and correctly executed 977 of them, compared with 491 and 187 for the baseline. It generated 985 properties, 912 of which were valid, and repaired 118 of 127 imprecise ones exposed during testing. With the resulting properties, we found 25 previously unknown functional bugs in the latest versions of the subject apps, many of which were missed by existing functional testing techniques.

Related papers

• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• Do Phone-Use Agents Respect Your Privacy? [97.81424230136075]
  We introduce MyPhoneBench, a verifiable evaluation framework for privacy behavior in mobile agents.<n>We operationalize privacy-respecting phone use as permissioned access, minimal disclosure, and user-controlled memory through a minimal privacy contract.<n>Across five frontier models on 10 mobile apps and 300 tasks, we find that task success, privacy-compliant task completion, and later-session use of saved preferences are distinct capabilities.
  arXiv  Detail & Related papers  (2026-04-01T14:50:50Z)
• FuncDroid: Towards Inter-Functional Flows for Comprehensive Mobile App GUI Testing [6.346121677855558]
  We introduce an inter-functional-flow-oriented GUI testing approach with the dual goals of precise model construction and deep bug detection.<n>By combining two complementary test-generation views, it can adaptively refine functional boundaries and systematically explore inter-functional flows.<n>We implement our approach in a tool called FuncDroid, and evaluate it on two benchmarks: (1) a widely-used open-source benchmark with 50 reproducible crash bugs and (2) a diverse set of 52 popular commercial apps.
  arXiv  Detail & Related papers  (2026-02-13T11:40:02Z)
• Screencast-Based Analysis of User-Perceived GUI Responsiveness [53.53923672866705]
  tool is a technique that measures GUI responsiveness directly from mobile screencasts.<n>It uses computer vision to detect user interactions and analyzes frame-level visual changes to compute two key metrics.<n>tool has been deployed in an industrial testing pipeline and analyzes thousands of screencasts daily.
  arXiv  Detail & Related papers  (2025-08-02T12:13:50Z)
• A Study of Using Multimodal LLMs for Non-Crash Functional Bug Detection in Android Apps [6.490384059173902]
  This study empirically investigates the capability of leveraging large language models (LLMs) to be test oracles to detect non-crash functional (NCF) bugs in Android apps. We conducted a comprehensive empirical study to explore the effectiveness of LLMs for detecting NCF bugs in Android apps on 71 well-documented NCF bugs. We successfully detected 24 previously unknown NCF bugs in 64 Android apps, with four of these bugs being confirmed or fixed.
  arXiv  Detail & Related papers  (2024-07-26T19:18:35Z)
• Same App, Different Behaviors: Uncovering Device-specific Behaviors in Android Apps [16.015694809763538]
  We present the first large-scale empirical study of device-specific behaviors in real-world Android apps. By examining the distribution of device-specific behaviors, our analysis revealed that apps within the Chinese third-party app market exhibit more relevant behaviors compared to their counterparts in Google Play. Beyond common behaviors like issue fixes and feature adaptations, we observed 33 aggressive apps, including popular ones with millions of downloads.
  arXiv  Detail & Related papers  (2024-06-14T07:58:22Z)
• GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection? [50.53312866647302]
  HateCheck is a suite for testing fine-grained model functionalities on synthesized data. We propose GPT-HateCheck, a framework to generate more diverse and realistic functional tests from scratch. Crowd-sourced annotation demonstrates that the generated test cases are of high quality.
  arXiv  Detail & Related papers  (2024-02-23T10:02:01Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• Multi-Objective Improvement of Android Applications [10.660480034605243]
  We write tests for 21 versions of 7 Android apps, creating a new benchmark for performance improvements. We use Genetic improvement, a search-based technique that navigates the space of software variants to find improved software. We were able to improve execution time by up to 35%, and memory use by up to 33% in these apps.
  arXiv  Detail & Related papers  (2023-08-22T12:26:43Z)
• A statistical approach for finding property-access errors [2.4171019220503402]
  We study the problem of finding incorrect property accesses in JavaScript where objects do not have a fixed layout. Since referencing a non-existent property is not an error in JavaScript, accidental accesses to non-existent properties can go undetected. We propose a two-phase approach for detecting property access errors based on the observation that, in practice, most property accesses will be correct.
  arXiv  Detail & Related papers  (2023-06-14T20:50:12Z)
• Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing [66.57888248681303]
  We propose a novel emerging issue detection approach named MERIT. Based on the AOBST model, we infer the topics negatively reflected in user reviews for one app version. Experiments on popular apps from Google Play and Apple's App Store demonstrate the effectiveness of MERIT.
  arXiv  Detail & Related papers  (2020-08-23T06:34:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.