Grounding World Simulation Models in a Real-World Metropolis

• URL: http://arxiv.org/abs/2603.15583v1
• Date: Mon, 16 Mar 2026 17:46:04 GMT
• Title: Grounding World Simulation Models in a Real-World Metropolis
• Abstract summary: We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul.<n>SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images.<n>We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor.
• Score: 80.10324496369951
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: What if a world simulation model could render not an imagined environment but a city that actually exists? Prior generative world models synthesize visually plausible yet artificial environments by imagining all content. We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul. SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images. However, this design introduces several challenges, including temporal misalignment between retrieved references and the dynamic target scene, limited trajectory diversity and data sparsity from vehicle-mounted captures at sparse intervals. We address these challenges through cross-temporal pairing, a large-scale synthetic dataset enabling diverse camera trajectories, and a view interpolation pipeline that synthesizes coherent training videos from sparse street-view images. We further introduce a Virtual Lookahead Sink to stabilize long-horizon generation by continuously re-grounding each chunk to a retrieved image at a future location. We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor. SWM outperforms existing methods in generating spatially faithful, temporally consistent, long-horizon videos grounded in actual urban environments over trajectories reaching hundreds of meters, while supporting diverse camera movements and text-prompted scenario variations.

Related papers

• Worlds in One Demo: A Synthetic Data Engine for Learning Open-World Mobile Manipulation [7.87892459538874]
  We introduce WANDA: learning open-World mobile mANipulation from one demonstration via a synthetic DAta engine.<n>We apply Corrective State Expansion to increase the robot and object state diversity at different stages of mobile manipulation.<n>Experiments show that policies trained with WANDA achieve long-horizon robustness, broad spatial generalization and cross-environment generalization.
  arXiv  Detail & Related papers  (2026-07-14T18:04:58Z)
• WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence [31.53753201666142]
  We introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments.<n>Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception.<n>We systematically analyze the key challenges on the path to simulation-ready urban digital twins.
  arXiv  Detail & Related papers  (2026-07-07T22:19:51Z)
• CityRAG: Stepping Into a City via Spatially-Grounded Video Generation [61.843558834675065]
  We present CityRAG, a video generative model that generates a 3D-consistent, navigable environment that is spatially grounded.<n>CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes.<n>Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
  arXiv  Detail & Related papers  (2026-04-21T17:59:03Z)
• SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Model [30.561378506172698]
  We propose SceneDiffuser++, the first end-to-end generative world model trained on a single loss function capable of point A-to-B simulation on a city scale.<n>We demonstrate the city-scale traffic simulation capability of SceneDiffuser++ and study its superior realism under long simulation conditions.
  arXiv  Detail & Related papers  (2025-06-27T07:35:04Z)
• Learning 3D Persistent Embodied World Models [84.40585374179037]
  We introduce a new persistent embodied world model with an explicit memory of previously generated content.<n>During generation time, our video diffusion model predicts RGB-D video of the future observations of the agent.<n>This generation is then aggregated into a persistent 3D map of the environment.
  arXiv  Detail & Related papers  (2025-05-05T17:59:17Z)
• SimVS: Simulating World Inconsistencies for Robust View Synthesis [102.83898965828621]
  We present an approach for leveraging generative video models to simulate the inconsistencies in the world that can occur during capture.<n>We demonstrate that our world-simulation strategy significantly outperforms traditional augmentation methods in handling real-world scene variations.
  arXiv  Detail & Related papers  (2024-12-10T17:35:12Z)
• The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control [16.075784652681172]
  We present The Matrix, the first foundational realistic world simulator capable of generating continuous 720p real-scene video streams.<n>The Matrix allows users to traverse diverse terrains in continuous, uncut hour-long sequences.<n>The Matrix can simulate a BMW X3 driving through an office setting--an environment present in neither gaming data nor real-world sources.
  arXiv  Detail & Related papers  (2024-12-04T18:59:05Z)
• OmniRe: Omni Urban Scene Reconstruction [78.99262488964423]
  We introduce OmniRe, a comprehensive system for creating high-fidelity digital twins of dynamic real-world scenes from on-device logs.<n>Our approach builds scene graphs on 3DGS and constructs multiple Gaussian representations in canonical spaces that model various dynamic actors.
  arXiv  Detail & Related papers  (2024-08-29T17:56:33Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• Future Urban Scenes Generation Through Vehicles Synthesis [90.1731992199415]
  We propose a deep learning pipeline to predict the visual future appearance of an urban scene. We follow a two stages approach, where interpretable information is included in the loop and each actor is modelled independently. We show the superiority of this approach over traditional end-to-end scene-generation methods on CityFlow.
  arXiv  Detail & Related papers  (2020-07-01T08:40:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.