Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

• URL: http://arxiv.org/abs/2607.28227v1
• Date: Thu, 30 Jul 2026 13:58:41 GMT
• Title: Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents
• Abstract summary: Qwen-UI-Agent is a real-world centric foundation GUI agent spanning mobile, computer-use, web, and DeepSearch environments.<n>Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn.<n>Online RL supports training on trajectories exceeding 100 turns, with over 10,000 environments accelerating rollout.
• Score: 27.09949595397669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision agents that operate reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI execution, complete long-horizon tasks, proactively initiate useful services, and autonomously improve their capabilities with minimal human effort. Guided by this vision, we present Qwen-UI-Agent, a real-world centric foundation GUI agent spanning mobile, computer-use, web, and DeepSearch environments. Qwen-UI-Agent combines diverse sandbox environments with a large-scale real-device mobile runtime. Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn. An AutoResearch-style data flywheel uses agents to construct tasks and environments, diagnose failures, and plan subsequent iterations. Online RL supports training on trajectories exceeding 100 turns, with over 10,000 concurrent environments accelerating rollout. A lightweight harness layer supports proactive service initiation and stateful workflows across mobile and computer. Across a broad suite of evaluations, Qwen-UI-Agent sets state-of-the-art performance on mobile-use benchmarks while delivering competitive performance on computer- and browser-use tasks against frontier models, including Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol. On mobile use, it achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. On computer use, it achieves 79.5% on OSWorld-Verified and a 40.0% partial-progress score on OSWorld-v2. On browser use and GUI grounding, it achieves 73.6% on WebArena and 81.5% on ScreenSpot-Pro, respectively.

Related papers

• UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations [68.29650478987561]
  UI-Mate is a foundation GUI agent that integrates an environment-grounded training stack with in-context demonstration learning.<n>It sets a new open-weight state of the art on general computer-use benchmarks, scoring 77.0% on OSWorld and 66.2% on WindowsAgentArena.
  arXiv  Detail & Related papers  (2026-08-16T20:59:59Z)
• Qwen-CUA: Native Computer Use for (almost) Everything [104.78368489343713]
  We introduce Qwen-CUA, a native computer-use agent with a mixture-of-experts backbone.<n>It observes only screenshots and acts through keyboard and mouse events, without DOM trees, accessibility metadata, or task-specific APIs.<n>Its scaffold maintains up to 20 active screenshots and folds older visual history in fixed-size blocks to retain recent evidence.
  arXiv  Detail & Related papers  (2026-08-03T15:04:20Z)
• Beyond the GUI Paradigm: Do Mobile Agents Need the Phone Screen? [23.855513024800526]
  Mobile platforms expose a command-line interface (CLI) that provides direct access to device services and data.<n>We evaluate three coding agents across four model APIs on AndroidWorld and MobileWorld without any mobile-specific post-training.<n>We provide oracle CLI solutions that reach 88.8% on AndroidWorld (103/116 tasks CLI-solvable) and 86.3% on MobileWorld (101/117 tasks CLI-solvable)<n>To support future research on mobile CLI agents, we will open agent implementations, oracle solutions, the CLI-Advantage suite, and evaluation infrastructure.
  arXiv  Detail & Related papers  (2026-06-16T02:36:22Z)
• ProCUA-SFT Technical Report [80.97543110323542]
  Training computer-use agents (CUAs) requires large-scale, diverse trajectory data collected in full desktop environments.<n>We present ProCUA-SFT, a dataset of 3.1M step-level SFT samples distilled from 93K synthetic trajectories.<n>Fine-tuning UI-TARS 7B on ProCUA-SFT for one epoch yields 45.0% on OSWorld -- an 18.7 percentage-point improvement over the base model.
  arXiv  Detail & Related papers  (2026-06-15T22:04:11Z)
• Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents [56.72789202127874]
  The paper introduces GUI-Owl-1.5, the latest native GUI agent model.<n>It supports a range of platforms (desktop, mobile, browser, and more) to enable cloud-edge collaboration and real-time interaction.<n>It achieves state-of-the-art results on more than 20+ GUI benchmarks on open-source models.
  arXiv  Detail & Related papers  (2026-02-15T01:52:19Z)
• MAI-UI Technical Report: Real-World Centric Foundation GUI Agents [33.46555542782679]
  MAI-UI is a family of foundation GUI agents spanning the full spectrum of sizes, including 2B, 8B, 32B, and 235B-A22B variants.<n>We identify four key challenges to realistic deployment: the lack of native agent-user interaction, the limits of UI-only operation, and the absence of a practical deployment architecture.
  arXiv  Detail & Related papers  (2025-12-26T14:51:52Z)
• UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning [155.51875080423883]
  The development of autonomous agents for graphical user interfaces presents major challenges in artificial intelligence.<n>We present UI-TARS-2, a native GUI-centered agent model that addresses these challenges through a systematic training methodology.<n> Empirical evaluation demonstrates that UI-TARS-2 achieves significant improvements over its predecessor UI-TARS-1.5.
  arXiv  Detail & Related papers  (2025-09-02T17:44:45Z)
• Mobile-Agent-v3: Fundamental Agents for GUI Automation [59.775510710011325]
  This paper introduces a foundational GUI agent model that achieves state-of-the-art performance among open-source end-to-end models.<n>We propose Mobile-Agent-v3, a general-purpose GUI agent framework that further improves performance to 73.3 on AndroidWorld and 37.7 on OSWorld.
  arXiv  Detail & Related papers  (2025-08-21T00:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.