WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence

• URL: http://arxiv.org/abs/2607.06838v1
• Date: Tue, 07 Jul 2026 22:19:51 GMT
• Title: WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence
• Abstract summary: We introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments.<n>Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception.<n>We systematically analyze the key challenges on the path to simulation-ready urban digital twins.
• Score: 31.53753201666142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable scale? Although recent foundation models have advanced scene reconstruction and embodied intelligence, scaling to entire cities remains an open challenge, primarily due to the lack of city-scale data. To bridge the gap, we introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments. Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception, e.g., dynamic objects, lighting variations, and imperfect camera poses. We further establish an urban-tailored reconstruction baseline and convert the reconstructed environments into a closed-loop simulator. Beyond the dataset and baseline, we systematically analyze the key challenges on the path to simulation-ready urban digital twins: scalability, extrapolation, and uncertainty. Ultimately, WildCity aims to catalyze progress not only in city-scale rendering, but more broadly in the pursuit of AI that can perceive, remember, and reason across space at a scale comparable to human cognition. Project page: https://han-xiangyu.github.io/Wild-City/

Related papers

• CityRAG: Stepping Into a City via Spatially-Grounded Video Generation [61.843558834675065]
  We present CityRAG, a video generative model that generates a 3D-consistent, navigable environment that is spatially grounded.<n>CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes.<n>Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
  arXiv  Detail & Related papers  (2026-04-21T17:59:03Z)
• Grounding World Simulation Models in a Real-World Metropolis [80.10324496369951]
  We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul.<n>SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images.<n>We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor.
  arXiv  Detail & Related papers  (2026-03-16T17:46:04Z)
• TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding [12.573182815543978]
  3D semantic scene understanding remains a long-standing challenge in the 3D computer vision community.<n>We introduce TrueCity, the first urban semantic segmentation benchmark with cm-accurate annotated real-world point clouds, semantic 3D city models, and annotated simulated point clouds representing the same city.
  arXiv  Detail & Related papers  (2025-11-10T11:57:50Z)
• UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos [64.22243628420799]
  We introduce UrbanVerse, a data-driven real-to-sim system that converts crowd-sourced city-tour videos into physics-aware, interactive simulation scenes.<n>Running in IsaacSim, UrbanVerse offers 160 high-quality constructed scenes from 24 countries, along with a curated benchmark of 10 artist-designed test scenes.<n>Experiments show that UrbanVerse scenes preserve real-world semantics and layouts, achieving human-evaluated realism comparable to manually crafted scenes.
  arXiv  Detail & Related papers  (2025-10-16T17:42:34Z)
• SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Model [30.561378506172698]
  We propose SceneDiffuser++, the first end-to-end generative world model trained on a single loss function capable of point A-to-B simulation on a city scale.<n>We demonstrate the city-scale traffic simulation capability of SceneDiffuser++ and study its superior realism under long simulation conditions.
  arXiv  Detail & Related papers  (2025-06-27T07:35:04Z)
• Compositional Generative Model of Unbounded 4D Cities [56.36624718397362]
  We propose a compositional generative model specifically tailored for generating 4D cities.<n>CityDreamer4D supports a range of downstream applications, such as instance editing, city stylization, and urban simulation.
  arXiv  Detail & Related papers  (2025-01-15T17:59:56Z)
• CityX: Controllable Procedural Content Generation for Unbounded 3D Cities [50.10101235281943]
  Current generative methods fall short in either diversity, controllability, or fidelity.<n>In this work, we resort to the procedural content generation (PCG) technique for high-fidelity generation.<n>We develop a multi-agent framework to transform multi-modal instructions, including OSM, semantic maps, and satellite images, into executable programs.<n>Our method, named CityX, demonstrates its superiority in creating diverse, controllable, and realistic 3D urban scenes.
  arXiv  Detail & Related papers  (2024-07-24T18:05:13Z)
• MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyond [69.37319723095746]
  We build a large-scale, comprehensive, and high-quality synthetic dataset for city-scale neural rendering researches. We develop a pipeline to easily collect aerial and street city views, accompanied by ground-truth camera poses and a range of additional data modalities. The resulting pilot dataset, MatrixCity, contains 67k aerial images and 452k street images from two city maps of total size $28km2$.
  arXiv  Detail & Related papers  (2023-09-28T16:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.