CityRAG: Stepping Into a City via Spatially-Grounded Video Generation

• URL: http://arxiv.org/abs/2604.19741v1
• Date: Tue, 21 Apr 2026 17:59:03 GMT
• Title: CityRAG: Stepping Into a City via Spatially-Grounded Video Generation
• Abstract summary: We present CityRAG, a video generative model that generates a 3D-consistent, navigable environment that is spatially grounded.<n>CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes.<n>Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
• Score: 61.843558834675065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the problem of generating a 3D-consistent, navigable environment that is spatially grounded: a simulation of a real location. Existing video generative models can produce a plausible sequence that is consistent with a text (T2V) or image (I2V) prompt. However, the capability to reconstruct the real world under arbitrary weather conditions and dynamic object configurations is essential for downstream applications including autonomous driving and robotics simulation. To this end, we present CityRAG, a video generative model that leverages large corpora of geo-registered data as context to ground generation to the physical scene, while maintaining learned priors for complex motion and appearance changes. CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes. Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.

Related papers

• Lyra 2.0: Explorable Generative 3D Worlds [77.45279013687427]
  We present Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale.<n>To address spatial forgetting, we maintain per-frame 3D geometry and use it solely for information routing.<n>We train with self-augmented histories that expose the model to its own degraded outputs, teaching it to correct drift rather than propagate it.
  arXiv  Detail & Related papers  (2026-04-14T17:59:44Z)
• Grounding World Simulation Models in a Real-World Metropolis [80.10324496369951]
  We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul.<n>SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images.<n>We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor.
  arXiv  Detail & Related papers  (2026-03-16T17:46:04Z)
• Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes [22.450051108066216]
  We present C2R (Coarse-to-Real), a generative framework that synthesizes real-style urban crowd videos.<n>Our approach uses coarse 3D renderings to explicitly control scene layout, camera motion, and human trajectories.<n>It produces temporally consistent, controllable, and realistic urban scene videos from minimal 3D input.
  arXiv  Detail & Related papers  (2026-01-29T20:29:04Z)
• Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
  We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
  arXiv  Detail & Related papers  (2026-01-24T07:43:57Z)
• Learning 3D-Gaussian Simulators from RGB Videos [20.250137125726265]
  3DGSim is a learned 3D simulator that learns physical interactions from multi-view RGB videos.<n>It unifies 3D scene reconstruction, particle dynamics prediction and video synthesis into an end-to-end trained framework.
  arXiv  Detail & Related papers  (2025-03-31T12:33:59Z)
• Stag-1: Towards Realistic 4D Driving Simulation with Video Generation Model [83.31688383891871]
  We propose a Spatial-Temporal simulAtion for drivinG (Stag-1) model to reconstruct real-world scenes.<n>Stag-1 constructs continuous 4D point cloud scenes using surround-view data from autonomous vehicles.<n>It decouples spatial-temporal relationships and produces coherent driving videos.
  arXiv  Detail & Related papers  (2024-12-06T18:59:56Z)
• Urban Scene Diffusion through Semantic Occupancy Map [49.20779809250597]
  UrbanDiffusion is a 3D diffusion model conditioned on a Bird's-Eye View (BEV) map. Our model learns the data distribution of scene-level structures within a latent space. After training on real-world driving datasets, our model can generate a wide range of diverse urban scenes.
  arXiv  Detail & Related papers  (2024-03-18T11:54:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.