Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving

• URL: http://arxiv.org/abs/2605.18137v5
• Date: Wed, 27 May 2026 13:09:09 GMT
• Title: Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving
• Abstract summary: This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving.<n>For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries.<n>For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning.
• Score: 51.90209659403234
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving: world representation and world generation. For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries. WorldRec initializes structured queries in 3D space, leveraging them to aggregate cross-view, cross-temporal features, thereby naturally enforcing spatial consistency across frames and yielding compact yet high-fidelity 3D Gaussian scene representations. For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning through three progressive stages (Teacher Forcing, ODE distillation, and DMD), enabling high-quality online causal video generation in as few as 4 denoising steps. Building on both modules, we further introduce the JWM, which deeply integrates WorldRec and WorldGen to achieve synergistic gains in generation stability, cross-frame consistency, and visual fidelity, providing a solid foundation for closed-loop simulation, data synthesis, and end-to-end training in autonomous driving.

Related papers

• ReWorld: Learning Better Representations for World Action Models [41.92917169094194]
  World Action Models (WAMs) model future environment evolution under action conditioning.<n>We propose ReWorld, the first representation learning framework specifically designed for WAMs.<n>We show that ReWorld improves fine-tuned video generation by 23.9% in FVD.
  arXiv  Detail & Related papers  (2026-06-25T19:37:58Z)
• World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Tasks [62.389116510844445]
  We introduce emphWorld-Ego Modeling, a new conceptual paradigm that decomposes future evolution into world and ego components.<n>We instantiate this paradigm as the World-Ego Model (WEM), a unified embodied world model that couples an implicit separate world-ego planner with a cascade-parallel mixture-of-experts (CP-MoE) diffusion generator.
  arXiv  Detail & Related papers  (2026-05-19T15:10:27Z)
• Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception [3.3242611619309614]
  This paper presents a vision for Infrastructure-centric World Models (I-WM) in three phases.<n>We propose a dual-layer architecture, annotation-free perception as a multi-modal data engine feeding end-to-end generative world models.<n>We establish a taxonomy of driving world model paradigms, position I-WM relative to LeCun's JEPA, Li Fei-Fei's spatial intelligence, and VLA architectures.
  arXiv  Detail & Related papers  (2026-04-19T22:50:32Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• Causal World Modeling for Robot Control [56.31803788587547]
  Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
  arXiv  Detail & Related papers  (2026-01-29T17:07:43Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• DSG-World: Learning a 3D Gaussian World Model from Dual State Videos [14.213608866611784]
  We present DSG-World, a novel end-to-end framework that explicitly constructs a 3D Gaussian World model from Dual State observations.<n>Our approach builds dual segmentation-aware Gaussian fields and enforces bidirectional photometric and semantic consistency.
  arXiv  Detail & Related papers  (2025-06-05T16:33:32Z)
• A Survey of World Models for Autonomous Driving [55.520179689933904]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>World models offer high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics.<n>Future research must address key challenges in self-supervised representation learning, multimodal fusion, and advanced simulation.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey [61.39993881402787]
  World models and video generation are pivotal technologies in the domain of autonomous driving. This paper investigates the relationship between these two technologies. By analyzing the interplay between video generation and world models, this survey identifies critical challenges and future research directions.
  arXiv  Detail & Related papers  (2024-11-05T08:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.