X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving

• URL: http://arxiv.org/abs/2603.19979v1
• Date: Fri, 20 Mar 2026 14:23:32 GMT
• Title: X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving
• Abstract summary: We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space.<n>X-World supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control.<n>Experiments show that X-World achieves high-quality multi-view video generation with strong view consistency across cameras.
• Score: 45.260296714057766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scalable and reliable evaluation is increasingly critical in the end-to-end era of autonomous driving, where vision--language--action (VLA) policies directly map raw sensor streams to driving actions. Yet, current evaluation pipelines still rely heavily on real-world road testing, which is costly, biased toward limited scenario coverage, and difficult to reproduce. These challenges motivate a real-world simulator that can generate realistic future observations under proposed actions, while remaining controllable and stable over long horizons. We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space. Given synchronized multi-view camera history and a future action sequence, X-World generates future multi-camera video streams that follow the commanded actions. To ensure reproducible and editable scene rollouts, X-World further supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control (e.g., weather and time of day). Beyond world simulation, X-World also enables video style transfer by conditioning on appearance prompts while preserving the underlying action and scene dynamics. At the core of X-World is a multi-view latent video generator designed to explicitly encourage cross-view geometric consistency and temporal coherence under diverse control signals. Experiments show that X-World achieves high-quality multi-view video generation with (i) strong view consistency across cameras, (ii) stable temporal dynamics over long rollouts, and (iii) high controllability with strict action following and faithful adherence to optional scene controls. These properties make X-World a practical foundation for scalable and reproducible evaluation.

Related papers

• M$^\ ext{4}$World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming [26.912855667055307]
  M$text4$World is a Multi-view and Multimodal generative driving world model.<n>Fine-grained object manipulation is realized through a flexible conditioning interface.<n>Stable minute-long streaming is achieved through a multi-stage training framework.
  arXiv  Detail & Related papers  (2026-07-15T16:36:33Z)
• WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory [106.28234880163723]
  WorldDirector is a highly controllable video world model framework.<n>Our framework explicitly decouples semantic motion orchestration from visual generation.<n>We successfully preserve the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.
  arXiv  Detail & Related papers  (2026-07-02T17:59:59Z)
• Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control [56.828569670698975]
  "Directing the World" is a fast autoregressive framework for controllable world-model video generation with compositional human-motion and camera-trajectory control.<n>Our key idea is to decouple control learning while preserving a unified autoregressive video prior.<n>For human motion control, we design a t-guided Dynamic Projection mechanism and a refined Motion-CFG strategy.
  arXiv  Detail & Related papers  (2026-06-26T11:08:09Z)
• DisCo: World Models with Discrete Camera Motion Control [79.86256515640231]
  We propose DisCo, a controllable video world model that conditions generation on a compact set of discrete action primitives to improve action separability.<n>DisCo achieves significantly more reliable action following while preserving visual quality.
  arXiv  Detail & Related papers  (2026-06-06T03:50:45Z)
• MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
  MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
  arXiv  Detail & Related papers  (2026-06-01T18:20:20Z)
• AnyScene: Towards Highly Controllable Driving Scene Generation at Anywhere and Beyond [34.2135689224471]
  AnyScene is a unified occupancy-centric framework for driving scene generation.<n>It generates semantic occupancy sequences from BEV layouts through a Spatial-Temporal Occupancy Diffusion Transformer.<n>Building upon the generated occupancy, a Geometry-Grounded View Expansion module treats occupancy as the canonical spatial representation.
  arXiv  Detail & Related papers  (2026-05-25T17:59:48Z)
• LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models [32.92934803081681]
  Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control.<n>They implicitly assume that the world only evolves within the observer's field of view.<n>Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime.<n>We propose LiveWorld, a novel framework that extends video world models to support persistent world evolution.
  arXiv  Detail & Related papers  (2026-03-07T10:31:39Z)
• DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving [49.11389494068169]
  We present DrivingGen, the first comprehensive benchmark for generative driving world models.<n>DrivingGen combines a diverse evaluation dataset curated from both driving datasets and internet-scale video sources.<n>General models look better but break physics, while driving-specific ones capture motion realistically but lag in visual quality.
  arXiv  Detail & Related papers  (2026-01-04T13:36:21Z)
• Astra: General Interactive World Model with Autoregressive Denoising [73.6594791733982]
  Astra is an interactive general world model that generates real-world futures for diverse scenarios.<n>We propose an autoregressive denoising architecture and use temporal causal attention to aggregate past observations.<n>Astra achieves interactive, consistent, and general long-term video prediction and supports various forms of interactions.
  arXiv  Detail & Related papers  (2025-12-09T18:59:57Z)
• Controllable Pedestrian Video Editing for Multi-View Driving Scenarios via Motion Sequence [4.778564042492516]
  We present a framework for controllable pedestrian video editing in multi-view driving scenarios by integrating video inpainting and human motion control techniques.<n>Our approach begins by identifying pedestrian regions of interest across multiple camera views, expanding detection bounding boxes with a fixed ratio, and resizing and stitching these regions into a unified canvas.<n>Experiments demonstrate that our framework achieves high-quality pedestrian editing with strong visual realism coherence, and cross-view consistency.
  arXiv  Detail & Related papers  (2025-08-01T03:56:57Z)
• X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability [49.4647778989539]
  X-Scene is a novel framework for large-scale driving scene generation.<n>It achieves both geometric intricacy and appearance fidelity, while offering flexible controllability.<n>X-Scene significantly advances controllability and fidelity for large-scale driving scene generation.
  arXiv  Detail & Related papers  (2025-06-16T14:43:18Z)
• Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention [61.3281618482513]
  We present CogDriving, a novel network designed for synthesizing high-quality multi-view driving videos.<n>CogDriving leverages a Diffusion Transformer architecture with holistic-4D attention modules, enabling simultaneous associations across the dimensions.<n>CogDriving demonstrates strong performance on the nuScenes validation set, achieving an FVD score of 37.8, highlighting its ability to generate realistic driving videos.
  arXiv  Detail & Related papers  (2024-12-04T18:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.