Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception
- URL: http://arxiv.org/abs/2604.17651v1
- Date: Sun, 19 Apr 2026 22:50:32 GMT
- Title: Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception
- Abstract summary: This paper presents a vision for Infrastructure-centric World Models (I-WM) in three phases.<n>We propose a dual-layer architecture, annotation-free perception as a multi-modal data engine feeding end-to-end generative world models.<n>We establish a taxonomy of driving world model paradigms, position I-WM relative to LeCun's JEPA, Li Fei-Fei's spatial intelligence, and VLA architectures.
- Score: 3.3242611619309614
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: World models, generative AI systems that simulate how environments evolve, are transforming autonomous driving, yet all existing approaches adopt an ego-vehicle perspective, leaving the infrastructure viewpoint unexplored. We argue that infrastructure-centric world models offer a fundamentally complementary capability: the bird's-eye, multi-sensor, persistent viewpoint that roadside systems uniquely possess. Central to our thesis is a spatio-temporal complementarity: fixed roadside sensors excel at temporal depth, accumulating long-term behavioral distributions including rare safety-critical events, while vehicle-borne sensors excel at spatial breadth, sampling diverse scenes across large road networks. This paper presents a vision for Infrastructure-centric World Models (I-WM) in three phases: (I) generative scene understanding with quality-aware uncertainty propagation, (II) physics-informed predictive dynamics with multi-agent counterfactual reasoning, and (III) collaborative world models for V2X communication via latent space alignment. We propose a dual-layer architecture, annotation-free perception as a multi-modal data engine feeding end-to-end generative world models, with a phased sensor strategy from LiDAR through 4D radar and signal phase data to event cameras. We establish a taxonomy of driving world model paradigms, position I-WM relative to LeCun's JEPA, Li Fei-Fei's spatial intelligence, and VLA architectures, and introduce Infrastructure VLA (I-VLA) as a novel unification of roadside perception, language commands, and traffic control actions. Our vision builds upon existing multi-LiDAR pipelines and identifies open-source foundations for each phase, providing a path toward infrastructure that understands and anticipates traffic.
Related papers
- Plug-and-Play Traffic Element Awareness for End-to-End Autonomous Driving [22.38391140804806]
We present the first systematic investigation of traffic element awareness for end-to-end autonomous driving.<n>We construct a unified research infrastructure by augmenting multiple public driving datasets with traffic-element annotations.<n>We evaluate this design across modern paradigms, including perception-prediction-planning pipelines, vision-language-action models (VLA), regression-based planners, diffusion-based policies, and trajectory-scoring frameworks.
arXiv Detail & Related papers (2026-08-18T17:29:51Z) - Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving [51.90209659403234]
This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving.<n>For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries.<n>For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning.
arXiv Detail & Related papers (2026-05-18T09:46:16Z) - Telecom World Models: Unifying Digital Twins, Foundation Models, and Predictive Planning for 6G [60.29349053952935]
We introduce the Telecom World Model(TWM) concept, an architecture for learned, action-conditioned, uncertainty-aware modeling of telecom system dynamics.<n>We propose a three-layer architecture, comprising a field world model for spatial environment prediction, a control/dynamics world model for action-conditioned Key Performance Indicator (KPI) trajectory prediction, and a telecom foundation model layer for intent translation and orchestration.
arXiv Detail & Related papers (2026-04-08T09:41:58Z) - SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving [52.02379432801349]
We propose SGDrive, a novel framework that structures the VLM's representation learning around driving-specific knowledge hierarchies.<n>Built upon a pre-trained VLM backbone, SGDrive decomposes driving understanding into a scene-agent-goal hierarchy that mirrors human driving cognition.
arXiv Detail & Related papers (2026-01-09T08:55:42Z) - All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles [7.863490977061713]
Autonomous Vehicles (AVs) are transforming the future of transportation through advances in intelligent perception, decision-making, and control systems.<n>Their success is tied to one core capability, reliable object detection in complex and multimodal environments.<n>Recent breakthroughs in Computer Vision (CV) and Artificial Intelligence (AI) have driven remarkable progress.<n>This survey bridges that gap by delivering a forward-looking analysis of object detection in AVs.
arXiv Detail & Related papers (2025-10-30T16:08:25Z) - Towards Intelligent Transportation with Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework [62.47416496137193]
We propose a surveillance video assisted federated digital twin (SV-FDT) framework to empower ITSs with pedestrians and vehicles in-the-loop.<n>The architecture consists of three layers: (i) the end layer, which collects traffic surveillance videos from multiple sources; (ii) the edge layer, responsible for semantic segmentation-based visual understanding, twin agent-based interaction modeling, and local digital twin system (LDTS) creation in local regions; and (iii) the cloud layer, which integrates LDTSs across different regions to construct a global DT model in realtime.
arXiv Detail & Related papers (2025-03-06T07:36:06Z) - A Survey of World Models for Autonomous Driving [55.520179689933904]
Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>World models offer high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics.<n>Future research must address key challenges in self-supervised representation learning, multimodal fusion, and advanced simulation.
arXiv Detail & Related papers (2025-01-20T04:00:02Z) - Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey [61.39993881402787]
World models and video generation are pivotal technologies in the domain of autonomous driving.
This paper investigates the relationship between these two technologies.
By analyzing the interplay between video generation and world models, this survey identifies critical challenges and future research directions.
arXiv Detail & Related papers (2024-11-05T08:58:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.