Remote-Sensing City Layout Extraction with MLLM

• URL: http://arxiv.org/abs/2608.16484v1
• Date: Mon, 17 Aug 2026 12:23:52 GMT
• Title: Remote-Sensing City Layout Extraction with MLLM
• Abstract summary: Code-as-City casts urban extraction from a single top-down image as constrained code generation with a large multimodal language model (MLLM)<n>Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry.<n>The complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union.
• Score: 6.085818625714531
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.

Related papers

• Semantic-Guided Progressive Object Removal with Gaussian Splatting [8.821363895807243]
  We propose a novel framework that integrates Semantic-guided Block Matching (SBM) and Region-Wise Progressive Refinement (RPR) for high-quality 3D object removal.<n> Experimental results demonstrate that our approach outperforms existing Gaussian-based methods in terms of perceptual quality and coherence in 3D object removal.
  arXiv  Detail & Related papers  (2026-07-05T07:11:48Z)
• Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps [77.63233146945718]
  We introduce Cog3DMap, a framework that recurrently constructs an explicit 3D memory from multi-view images.<n>Our framework enables direct reasoning over a spatially structured 3D map, achieving state-of-the-art performance on various spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-24T10:05:32Z)
• Large Spatial Model: End-to-end Unposed Images to Semantic 3D [79.94479633598102]
  Large Spatial Model (LSM) processes unposed RGB images directly into semantic radiance fields. LSM simultaneously estimates geometry, appearance, and semantics in a single feed-forward operation. It can generate versatile label maps by interacting with language at novel viewpoints.
  arXiv  Detail & Related papers  (2024-10-24T17:54:42Z)
• Multiview Scene Graph [7.460438046915524]
  A proper scene representation is central to the pursuit of spatial intelligence. We propose to build Multiview Scene Graphs (MSG) from unposed images. MSG represents a scene topologically with interconnected place and object nodes.
  arXiv  Detail & Related papers  (2024-10-15T02:04:05Z)
• TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes [67.5351491691866]
  We present a novel framework, dubbed TeMO, to parse multi-object 3D scenes and edit their styles. Our method can synthesize high-quality stylized content and outperform the existing methods over a wide range of multi-object 3D meshes.
  arXiv  Detail & Related papers  (2023-12-07T12:10:05Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• OmniCity: Omnipotent City Understanding with Multi-level and Multi-view Images [72.4144257192959]
  The paper presents OmniCity, a new dataset for omnipotent city understanding from multi-level and multi-view images. The dataset contains over 100K pixel-wise annotated images that are well-aligned and collected from 25K geo-locations in New York City. With the new OmniCity dataset, we provide benchmarks for a variety of tasks including building footprint extraction, height estimation, and building plane/instance/fine-grained segmentation.
  arXiv  Detail & Related papers  (2022-08-01T15:19:25Z)
• Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
  This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
  arXiv  Detail & Related papers  (2022-05-05T17:59:55Z)
• 3D Instance Segmentation of MVS Buildings [5.2517244720510305]
  We present a novel framework for instance segmentation of 3D buildings from Multi-view Stereo (MVS) urban scenes. The emphasis of this work lies in detecting and segmenting 3D building instances even if they are attached and embedded in a large and imprecise 3D surface model.
  arXiv  Detail & Related papers  (2021-12-18T11:12:38Z)
• Generative View Synthesis: From Single-view Semantics to Novel-view Images [38.7873192939574]
  Generative View Synthesis (GVS) can synthesize multiple photorealistic views of a scene given a single semantic map. We first lift the input 2D semantic map onto a 3D layered representation of the scene in feature space. We then project the layered features onto the target views to generate the final novel-view images.
  arXiv  Detail & Related papers  (2020-08-20T17:48:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.