LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents

• URL: http://arxiv.org/abs/2609.13287v1
• Date: Wed, 09 Sep 2026 05:53:39 GMT
• Title: LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents
• Abstract summary: We present LLaDA-UI, a 16.7B- parameter MoE-based, block-wise diffusion vision-language GUI agent.<n>We show that LLaDA-UI substantially outperforms Qwen2.5-VL-7B and surpasses Qwen3-VL-8B on four of six reported GUI benchmarks.
• Score: 52.61417375636918
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion large language models (dLLMs) achieve high decoding efficiency through block-parallel, arbitrary-order generation, making them attractive for latency-sensitive applications. GUI agents represent a natural testbed for this paradigm, as they must repeatedly perceive screen states and emit structured, spatially grounded actions in real time. However, whether dLLMs can be extended into capable multimodal GUI agents while preserving their parallel decoding advantage remains an open question. We present LLaDA-UI, a 16.7B-parameter MoE-based, block-wise diffusion vision-language GUI agent. LLaDA-UI follows a two-stage training pipeline: general multimodal pre-training aligns a native-resolution vision encoder with the LLaDA2.0-mini-base diffusion language backbone, followed by GUI-agent supervised fine-tuning on diverse mobile, desktop, web, and grounding data. Across widely adopted grounding benchmarks and navigation benchmarks spanning multiple platforms, LLaDA-UI substantially outperforms Qwen2.5-VL-7B and surpasses Qwen3-VL-8B on four of six reported GUI benchmarks. These results establish block-wise diffusion as a practical generative paradigm for multimodal GUI agents.

Related papers

• LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model [43.908068133557215]
  We present LLaDA2.0-Uni, a unified discrete diffusion large language model (dLLM) that supports multimodal understanding and generation.<n>Its architecture combines a fully semantic discrete tokenizer, a MoE-based dLLM backbone, and a diffusion decoder.
  arXiv  Detail & Related papers  (2026-04-22T17:20:42Z)
• Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding [11.148970971741273]
  We evaluate whether discrete vision-language models can serve as a viable alternative to AR models for graphical user interface grounding.<n>We adapt LLaDA-V for single-turn action and bounding-box prediction, framing the task as text generation from multimodal input.<n>We show that adapted diffusion model with hybrid masking consistently outperforms the linear-masked variant.
  arXiv  Detail & Related papers  (2026-03-27T09:34:07Z)
• GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents [19.27396264271709]
  GUI-CEval is the first comprehensive benchmark for Chinese mobile GUI agents built entirely on physical device environments.<n>It spans 201 mainstream apps across four device types and adopts a two-level structure that evaluates both atomic abilities and realistic application-level performance along five dimensions: perception, planning, reflection, execution, and evaluation.
  arXiv  Detail & Related papers  (2026-03-16T09:45:33Z)
• PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents [151.86841216364294]
  We propose textbfPAL-UI (textbfPlanning with textbfActive textbfLook-back), a novel framework that enables GUI agents to adaptively retrieve past observations when required.<n> PAL-UI combines a dual-level summarization agent, capturing both observation-level cues and action-level outcomes, with a dedicated retrieval tool.
  arXiv  Detail & Related papers  (2025-10-01T01:48:39Z)
• SparkUI-Parser: Enhancing GUI Perception with Robust Grounding and Parsing [13.521180435948791]
  We propose a novel end-to-end framework for GUI perception.<n>Instead of using probability-based discrete modeling, we perform continuous modeling of coordinates.<n>This effectively mitigates the limitations inherent in the discrete output characteristics.
  arXiv  Detail & Related papers  (2025-09-05T08:24:12Z)
• Manager: Aggregating Insights from Unimodal Experts in Two-Tower VLMs and MLLMs [61.903626952650605]
  Two-Tower Vision--Language Models (VLMs) have demonstrated strong performance across various downstream VL tasks.<n>We propose Manager, a lightweight, efficient and effective plugin that adaptively aggregates insights from different levels of pre-trained unimodal experts.
  arXiv  Detail & Related papers  (2025-06-13T07:16:41Z)
• AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning [82.42421823672954]
  AgentCPM-GUI is built for robust and efficient on-device GUI interaction.<n>Our training pipeline includes grounding-aware pre-training to enhance perception.<n>AgentCPM-GUI achieves state-of-the-art performance on five public benchmarks.
  arXiv  Detail & Related papers  (2025-06-02T07:30:29Z)
• LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning [71.98260064022452]
  We introduce LLaDA-V, a purely diffusion-based Multimodal Large Language Model (MLLM) that integrates visual instruction tuning with masked diffusion models.<n>Built upon LLaDA, a representative large language diffusion model, LLaDA-V incorporates a vision encoder and connector that projects visual features into the language embedding space.
  arXiv  Detail & Related papers  (2025-05-22T17:23:26Z)
• InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection [38.833925781308665]
  We introduce textitInfiGUIAgent, an MLLM-based GUI Agent trained with a two-stage supervised fine-tuning pipeline.<n> Stage 1 enhances fundamental skills such as GUI understanding and grounding, while Stage 2 integrates hierarchical reasoning and expectation-reflection reasoning skills.<n>textitInfiGUIAgent achieves competitive performance on several GUI benchmarks.
  arXiv  Detail & Related papers  (2025-01-08T15:45:21Z)
• Liquid: Language Models are Scalable and Unified Multi-modal Generators [112.71734051183726]
  Liquid is an auto-regressive generation paradigm that seamlessly integrates visual comprehension and generation.<n>Unlike previous multimodal large language model (MLLM), Liquid achieves this integration using a single large language model.<n>For the first time, Liquid uncovers a scaling law that performance drop unavoidably brought by the unified training of visual and language tasks.
  arXiv  Detail & Related papers  (2024-12-05T16:48:16Z)
• ShowUI: One Vision-Language-Action Model for GUI Visual Agent [80.50062396585004]
  Building Graphical User Interface (GUI) assistants holds significant promise for enhancing human workflow productivity. We develop a vision-language-action model in digital world, namely ShowUI, which features the following innovations. ShowUI, a lightweight 2B model using 256K data, achieves a strong 75.1% accuracy in zero-shot screenshot grounding.
  arXiv  Detail & Related papers  (2024-11-26T14:29:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.