UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning

• URL: http://arxiv.org/abs/2602.01536v1
• Date: Mon, 02 Feb 2026 02:10:51 GMT
• Title: UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning
• Abstract summary: We present UniDWM, a unified driving world model that advances autonomous driving through multifaceted representation learning.<n>UniDWM constructs a structure- and dynamic-aware latent world representation that serves as a physically grounded state space.
• Score: 10.275940472665647
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Achieving reliable and efficient planning in complex driving environments requires a model that can reason over the scene's geometry, appearance, and dynamics. We present UniDWM, a unified driving world model that advances autonomous driving through multifaceted representation learning. UniDWM constructs a structure- and dynamic-aware latent world representation that serves as a physically grounded state space, enabling consistent reasoning across perception, prediction, and planning. Specifically, a joint reconstruction pathway learns to recover the scene's structure, including geometry and visual texture, while a collaborative generation framework leverages a conditional diffusion transformer to forecast future world evolution within the latent space. Furthermore, we show that our UniDWM can be deemed as a variation of VAE, which provides theoretical guidance for the multifaceted representation learning. Extensive experiments demonstrate the effectiveness of UniDWM in trajectory planning, 4D reconstruction and generation, highlighting the potential of multifaceted world representations as a foundation for unified driving intelligence. The code will be publicly available at https://github.com/Say2L/UniDWM.

Related papers

• NavWM: A Unified Navigation World Model for Foresight-Driven Planning [15.801647085785826]
  NavWM is a unified navigation world model that seamlessly integrates latent world reasoning, multimodal action prediction, and controllable visual generation.<n>To overcome the limitations of deterministic policies, we introduce an anchor-based multimodal trajectory forecasting framework.<n>Experiments across diverse datasets demonstrate that NavWM significantly advances the state-the-art.
  arXiv  Detail & Related papers  (2026-06-23T03:30:20Z)
• Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundations [35.521396983843864]
  We propose a framework for turning foundation-model embeddings into compact, task-sufficient world representations.<n> TC-WM linearly projects high-dimensional visual embeddings into a compact latent as the dynamic space.<n>We show that TC-WM suffices to identify the underlying task-centric latent factors up to a simple transformation.
  arXiv  Detail & Related papers  (2026-05-25T09:21:43Z)
• Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation [68.03746493619285]
  We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing.<n>We build a scalable training recipe that combines unified instruction tuning, long-text rendering supervision, spatially grounded data, and both general and spatial editing signals.<n> Experiments across understanding, generation, long-text rendering, and editing benchmarks show that JoyAI-Image achieves state-of-the-art or highly competitive performance.
  arXiv  Detail & Related papers  (2026-05-05T15:49:47Z)
• HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation [83.31948299340782]
  HERMES++ is a unified driving world model that integrates 3D scene understanding and future geometry prediction within a single framework.<n>Hermes++ achieves strong performance, outperforming specialist approaches in both future point cloud prediction and 3D scene understanding tasks.
  arXiv  Detail & Related papers  (2026-04-30T17:59:58Z)
• Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation [66.7879424097418]
  We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying vision and motion representation.<n>A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories.<n>Experiments on the NAVSIM, NAVSIM-v2, and nuScenes benchmarks demonstrate that WorldDrive achieves leading planning performance among vision-only methods.
  arXiv  Detail & Related papers  (2026-03-16T07:59:39Z)
• Beyond Language Modeling: An Exploration of Multimodal Pretraining [125.34714978184638]
  We provide empirical clarity through controlled, from-scratch pretraining experiments.<n>We adopt the Transfusion framework, using next-token prediction for language and diffusion for vision.<n>We demonstrate that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language.
  arXiv  Detail & Related papers  (2026-03-03T18:58:00Z)
• The Trinity of Consistency as a Defining Principle for General World Models [106.16462830681452]
  General World Models are capable of learning, simulating, and reasoning about objective physical laws.<n>We propose a principled theoretical framework that defines the essential properties requisite for a General World Model.<n>Our work establishes a principled pathway toward general world models, clarifying both the limitations of current systems and the architectural requirements for future progress.
  arXiv  Detail & Related papers  (2026-02-26T16:15:55Z)
• Unified World Models: Memory-Augmented Planning and Foresight for Visual Navigation [69.94565127141483]
  Current approaches separate navigation planning from visual world modeling, leading to state-action misalignment and limited adaptability.<n>We propose UniWM, a unified, memory-augmented world model integrating egocentric visual foresight and planning within a single multimodal autoregressive backbone.<n>We show that UniWM substantially improves navigation success rates by up to 30%, significantly reduces trajectory errors compared to strong baselines, and exhibits impressive zero-shot generalization on the unseen TartanDrive dataset.
  arXiv  Detail & Related papers  (2025-10-09T18:18:11Z)
• Can World Models Benefit VLMs for World Dynamics? [59.73433292793044]
  We investigate the capabilities when world model priors are transferred into Vision-Language Models.<n>We name our best-performing variant Dynamic Vision Aligner (DyVA)<n>We find DyVA to surpass both open-source and proprietary baselines, achieving state-of-the-art or comparable performance.
  arXiv  Detail & Related papers  (2025-10-01T13:07:05Z)
• GWM: Towards Scalable Gaussian World Models for Robotic Manipulation [53.51622803589185]
  We propose a novel branch of world model named Gaussian World Model (GWM) for robotic manipulation.<n>At its core is a latent Diffusion Transformer (DiT) combined with a 3D variational autoencoder, enabling fine-grained scene-level future state reconstruction.<n>Both simulated and real-world experiments depict that GWM can precisely predict future scenes conditioned on diverse robot actions.
  arXiv  Detail & Related papers  (2025-08-25T02:01:09Z)
• TARDIS STRIDE: A Spatio-Temporal Road Image Dataset and World Model for Autonomy [44.85881816317044]
  We show how to permuting 360-degree panoramic imagery into rich interconnected observation, state and action nodes.<n>We benchmark this dataset via TARDIS, a transformer-based generative world model.<n>We demonstrate robust performance across a range of agentic tasks such as controllable image synthesis, instruction following, autonomous self-control, and state-of-the-art georeferencing.
  arXiv  Detail & Related papers  (2025-06-12T21:08:11Z)
• DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Driving [20.197094443215963]
  We present DriveX, a self-supervised world model that learns general scene dynamics and holistic representations from driving videos.<n>DriveX introduces Omni Scene Modeling (OSM), a module that unifies multimodal supervision-3D point cloud forecasting, 2D semantic representation, and image generation.<n>For downstream adaptation, we design Future Spatial Attention (FSA), a unified paradigm that dynamically aggregates features from DriveX's predictions to enhance task-specific inference.
  arXiv  Detail & Related papers  (2025-05-25T17:27:59Z)
• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving [19.81442567260658]
  We propose a visual-temporalT framework that enables VLAs to think in images.<n>On nuScenes and NAVSIM, FSDrive improves accuracy and reduces collisions.
  arXiv  Detail & Related papers  (2025-05-23T09:55:32Z)
• A Survey of World Models for Autonomous Driving [55.520179689933904]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>World models offer high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics.<n>Future research must address key challenges in self-supervised representation learning, multimodal fusion, and advanced simulation.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.