Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation

• URL: http://arxiv.org/abs/2606.17030v3
• Date: Wed, 17 Jun 2026 13:54:57 GMT
• Title: Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation
• Abstract summary: We introduce Qwen-RobotWorld, a language-conditioned video world model for embodied intelligence.<n>It predicts physically grounded future visual trajectories from current observations across robotic manipulation, autonomous driving, indoor navigation, and human-to-robot transfer.
• Score: 88.94027963091206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Qwen-RobotWorld, a language-conditioned video world model for embodied intelligence. With natural language as a unified action interface, it predicts physically grounded future visual trajectories from current observations across robotic manipulation, autonomous driving, indoor navigation, and human-to-robot transfer. This unified formulation provides three promising application directions: synthetic data generation for policy training augmentation, scalable virtual environments for policy evaluation, and language-guided planning signals for downstream robot control. This is achieved through a three-part design: a) Double-Stream MMDiT with MLLM Action Encoding, where a 60-layer double-stream diffusion transformer couples frozen Qwen2.5-VL semantics with video-VAE latents through layer-wise joint attention; b) Embodied World Knowledge (EWK), an 8.6M video-text corpus (200M+ frames) with action-language mapping over 20+ embodiments and 500+ action categories; and c) General+Expert Progressive Curriculum, a two-stage training strategy that first learns general visual priors and then injects embodied specialization under a shared language interface. Extensive results show strong competitiveness: ranks 1st overall on EWMBench and DreamGen Bench, outperforms all open-source models on WorldModelBench and PBench. Additional zero-shot analyses on RoboTwin-IF benchmark further support robust generalization and multi-view consistency.

Related papers

• iFLYTEK-Embodied-Omni Technical Report [30.02389585663246]
  General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons.<n>We present iFLYTEK-Embodied- Omni, a unified multimodal foundation model that jointly models vision(videos and images), language, and action within a single Omni framework.
  arXiv  Detail & Related papers  (2026-06-24T00:25:44Z)
• World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis [37.19737492128721]
  We propose world-language-action models as a new class of embodied foundation models.<n> WLA takes textual instructions, images, and robot states as inputs to jointly predict textual subtasks, subgoal images, and robot actions.<n>We show that WLA-0 achieves state-of-the-art multi-task and long-horizon learning abilities, e.g., 92.94% success rate on RoboTwin2.0 Clean and 56.5% success rate on RMBench.
  arXiv  Detail & Related papers  (2026-06-04T10:23:01Z)
• GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation [53.6619853878637]
  We introduce GE-Sim 2.0 (Genie Envisioner World Simulator 2.0), a closed-loop video world simulator for robotic manipulation.<n>Building on the action-conditioned video generation framework of Genie Envisioner, GE-Sim 2.0 is re-trained on thousands of hours of real-world robot data.<n>Three new modules close the loop from video simulation to policy learning.
  arXiv  Detail & Related papers  (2026-05-26T16:23:05Z)
• From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data [71.22409934108924]
  Human videos are abundant and capture rich interactions, providing diverse semantic and physical cues for real-world manipulation.<n>This survey provides a unified view of how human videos are transformed into effective knowledge for Vision-Language-Action (VLA) models.<n>We highlight three key open challenges in this area: structuring unstructured videos into training-ready episodes, grounding video-derived supervision into robot-executable actions under embodiment and viewpoint heterogeneity, and designing evaluation protocols that better predict real-world deployment performance and transfer efficiency.
  arXiv  Detail & Related papers  (2026-05-18T06:19:16Z)
• Multi-View Video Diffusion Policy: A 3D Spatio-Temporal-Aware Video Action Model [43.80385042752741]
  We introduce MV-VDP, a multiview video policy that jointly models 3D state of environment.<n> Extensive experiments show that MV-VDP enables data-efficient, robust, generalizable, and interpretable manipulation.<n> Experiments on Meta-World and real-world robotic platforms demonstrate that MV-VDP consistently outperforms video-prediction-based, 3D-based, and vision--language--action models.
  arXiv  Detail & Related papers  (2026-04-03T16:57:06Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• Causal World Modeling for Robot Control [56.31803788587547]
  Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
  arXiv  Detail & Related papers  (2026-01-29T17:07:43Z)
• iVideoGPT: Interactive VideoGPTs are Scalable World Models [70.02290687442624]
  World models empower model-based agents to interactively explore, reason, and plan within imagined environments for real-world decision-making. This work introduces Interactive VideoGPT, a scalable autoregressive transformer framework that integrates multimodal signals--visual observations, actions, and rewards--into a sequence of tokens. iVideoGPT features a novel compressive tokenization technique that efficiently discretizes high-dimensional visual observations.
  arXiv  Detail & Related papers  (2024-05-24T05:29:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.