MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

• URL: http://arxiv.org/abs/2605.26114v2
• Date: Wed, 27 May 2026 05:27:30 GMT
• Title: MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
• Abstract summary: MobileGym is a lightweight, fully controllable environment for everyday mobile use.<n>A layered state model and a declarative task-definition framework keep state programmability and task creation practical at scale.<n>A single programmatic judging mechanism delivers both deterministic evaluation verdicts and dense rewards.
• Score: 39.68394140712043
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present MobileGym, a browser-hosted, lightweight, fully controllable environment for everyday mobile use, targeting interaction fidelity without replicating proprietary backends. It enables two capabilities previously out of reach for everyday apps: verifiable outcome signals through deterministic state-based judging over structured JSON state, and scalable online RL through low-cost parallel rollouts. The full environment state is captured, configured, forked, and compared as structured JSON, and a single server can host hundreds of parallel instances, with about 400 MB memory per instance and about 3 s cold start. A layered state model and a declarative task-definition framework keep state programmability and task creation practical at scale, and a single programmatic judging mechanism delivers both deterministic evaluation verdicts and dense RL rewards. The accompanying MobileGym-Bench provides 416 parameterized task templates, including 256 test and 160 train templates, over 28 apps, with deterministic judges and a structured AnswerSheet protocol that avoids free-text matching failures. In a Sim-to-Real case study, GRPO on Qwen3-VL-4B-Instruct gains +12.8 percentage points on the 256-task test set, and on a 59-task real-device signal subset, real-device execution retains 95.1% of the simulation-side training gain. Project page: https://mobilegym.github.io.

Related papers

• CivBench: A Long-Horizon Benchmark for Tool-Mediated Agents in Civilization VI [15.345634962043858]
  CivBench is an open-source benchmark for evaluating language model agents in tool-mediated environments.<n>A single episode spans 300+ turns and produces thousands of tool calls over a large action space.<n>We use CivBench to characterise agent behaviour across four model families in 23 admissible runs.
  arXiv  Detail & Related papers  (2026-09-02T11:31:39Z)
• DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows [69.12339622053588]
  We introduce DuMateBench, a real-session benchmark reconstructed from anonymized and privacy-screened user sessions.<n>The resulting benchmark comprises 200 tasks spanning 8 broad scenarios and 17 fine-grained capability categories.<n>We execute these tasks in isolated Docker containers injected with three forms of real-world environmental complexity: Insufficient, Unstable, and Noisy.
  arXiv  Detail & Related papers  (2026-08-27T02:36:23Z)
• MobilePA-Bench: Benchmarking Mobile Planner Agents on Complex Real-World Tasks [37.73689537216401]
  We present textbfMobilePA-Bench, an interactive, stateful, and tool-centric benchmark for evaluating the tool-calling and planning abilities of mobile planning agents.<n>MobilePA-Bench runs on an executable sandbox that maintains live application databases and returns structured feedback, spanning $13$ functional domains and $212$ realistic mobile tools.<n>By pairing an interactive function-calling sandbox with evidence-based verification, MobilePA-Bench serves as both a practical diagnostic benchmark and an interactive foundation for agentic reinforcement learning.
  arXiv  Detail & Related papers  (2026-08-24T09:38:24Z)
• UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations [68.29650478987561]
  UI-Mate is a foundation GUI agent that integrates an environment-grounded training stack with in-context demonstration learning.<n>It sets a new open-weight state of the art on general computer-use benchmarks, scoring 77.0% on OSWorld and 66.2% on WindowsAgentArena.
  arXiv  Detail & Related papers  (2026-08-16T20:59:59Z)
• Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning [0.0]
  This report presents a proof-of-concept deployment of distributed NanoChat pretraining across two NVIDIA DGX Spark systems.<n>The cluster supports a 400-level AI course (CS 426) and a query engine for Security+ POGIL activities in CBS 255.
  arXiv  Detail & Related papers  (2026-08-07T13:41:57Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents [50.27505102982638]
  We introduce MM-ToolSandBox, a benchmark and evaluation framework for visually grounded tool-calling agents.<n>The framework provides a stateful execution environment spanning 500+ tools across 16 application domains.<n>It supports multi-image, multi-turn tasks where agents must ground progressively arriving visual inputs into executable tool calls.
  arXiv  Detail & Related papers  (2026-07-13T17:13:09Z)
• ISE: An Execution-Grounded Recipe for Multi-Turn OS-Agent Trajectories [18.739245125968175]
  Training capable OS agents requires data that simultaneously captures structured user intents, multi-turn task delegation, and grounded tool execution--properties absent from existing datasets.<n>We propose ISE (Intent -> Simulate -> Execute), a three-stage synthesis paradigm that addresses these gaps jointly.
  arXiv  Detail & Related papers  (2026-06-09T23:44:26Z)
• WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [88.10947115397971]
  This work presents WildClawBench, a native-runtime benchmark of 60 human-authored, bilingual, multimodal tasks spanning six thematic categories.<n>Each task averages roughly 8 minutes of wall-clock time and over 20 tool calls.<n> Grading is hybrid, combining deterministic rule-based checks, environment-state auditing of side effects, and an LLM/VLM judge for semantic verification.
  arXiv  Detail & Related papers  (2026-05-11T17:49:43Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models [0.42015796373418257]
  UNI is a discrete-event simulation framework for developing, benchmarking, and deploying distributed AI models.<n>It integrates seamlessly with PyTorch Distributed, enabling the same transition from simulation to real deployment.
  arXiv  Detail & Related papers  (2026-03-27T14:35:02Z)
• SKILLS: Structured Knowledge Injection for LLM-Driven Telecommunications Operations [0.0]
  General-purpose language model (LLM) agents execute telecom operations through real API interfaces, or do they require structured domain guidance?<n>We introduce SKILLS, a benchmark framework comprising 37 telecom operations scenarios spanning 8 TM Forum Open API domains.<n>Results across 5 open-weight model conditions and 185 scenario-runs show consistent skill lift across all models.
  arXiv  Detail & Related papers  (2026-03-16T14:48:53Z)
• SimpleTool: Parallel Decoding for Real-Time LLM Function Calling [21.7429929239065]
  SimpleTool achieves 3-6x end-to-end speedup (up to 9.6x) with only +8.2% parallelization overhead.<n>On Mobile Actions, ST-Qwen-0.5B outperforms Google's FunctionGemma in both accuracy and latency consistency.
  arXiv  Detail & Related papers  (2026-02-04T08:58:27Z)
• David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generation [1.6752458252726459]
  NL-to-Scenic generation with large language models (LLMs) suffers from scarce data, limited metrics.<n>We introduce NL2Scenic, an open dataset and framework with 146 NL/Scenic pairs, a difficulty-stratified 30-case test split, and an Example Retriever.<n>We evaluate 13 models: four proprietary (GPT-4o, GPT-5, Claude-Sonnet-4, Gemini-2.5-pro) and nine open-source code models (Qwen2.5Coder 0.5B-32B; CodeLlama 7B/13B/34B)
  arXiv  Detail & Related papers  (2025-10-15T21:37:02Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation [5.500769111819106]
  This paper investigates whether Large Language Models (LLMs) can perform the twofold task of slot and intent detection and natural language response generation for a smart home assistant.<n>We fine-tune LLMs to produce both action calls and text responses.<n>Our experiments show that 16-bit and 8-bit quantized variants preserve high accuracy on slot and intent detection and maintain a strong semantic coherence in generated text.
  arXiv  Detail & Related papers  (2025-02-18T15:03:17Z)
• Learning Cross-Modal Affinity for Referring Video Object Segmentation Targeting Limited Samples [61.66967790884943]
  Referring video object segmentation (RVOS) relies on sufficient data for a given scene. In more realistic scenarios, only minimal annotations are available for a new scene. We propose a model with a newly designed cross-modal affinity (CMA) module based on a Transformer architecture. CMA module builds multimodal affinity with a few samples, thus quickly learning new semantic information, and enabling the model to adapt to different scenarios.
  arXiv  Detail & Related papers  (2023-09-05T08:34:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.