SVI2LoD3: Agent-Driven Reconstruction of LoD3 Facade Openings in Semantic 3D City Models from Volunteered Street View Imagery using Large Language and Visual Models

• URL: http://arxiv.org/abs/2608.29992v1
• Date: Sun, 30 Aug 2026 19:37:01 GMT
• Title: SVI2LoD3: Agent-Driven Reconstruction of LoD3 Facade Openings in Semantic 3D City Models from Volunteered Street View Imagery using Large Language and Visual Models
• Abstract summary: This paper presents an end-to-end, agent-driven pipeline for the LoD3 reconstruction of facade openings in 3D city models.<n>We also introduce a novel evaluation metric for facade reconstruction, termed Facade Feature Distance (FFD)<n>The proposed pipeline and evaluation strategy together offer a practical and scalable contribution toward the automated generation and analysis of semantically enriched 3D city models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents an end-to-end, agent-driven pipeline for the LoD3 reconstruction of facade openings in 3D city models, producing directly usable CityGML-conform outputs. In contrast to existing approaches that rely on supervised semantic segmentation and therefore require large amounts of manually annotated training data, the proposed method employs a zero-shot segmentation strategy. This substantially reduces the annotation effort while still achieving strong performance in our benchmark on the eTRIMS dataset. A further key contribution is the enforcement of correct partonomic hierarchies, thereby producing CityGML-conform LoD3 building models. Beyond the reconstruction pipeline itself, this work also introduces a novel evaluation metric for facade reconstruction, termed Facade Feature Distance (FFD). Unlike conventional metrics such as mIoU or FRDS, which assess similarity primarily through pixel-wise overlap, FFD measures distance in a high-level feature space derived from a vision transformer. In doing so, it captures both semantic correctness and architectural layout, providing a more suitable assessment of facade reconstruction quality. The proposed pipeline and evaluation strategy together offer a practical and scalable contribution toward the automated generation and analysis of semantically enriched 3D city models. The developed code is published at: https://github.com/hcu-cml/citydb-SVI2LoD3-ai.

Related papers

• SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models [51.121529170856064]
  We propose SemCityLoc, a semantic-geometric alignment system that reframes aerial pose estimation.<n>Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models.<n>Experiments demonstrate substantial improvements over existing map-based approaches.
  arXiv  Detail & Related papers  (2026-06-25T18:13:39Z)
• LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows [10.300202521638274]
  We introduce the Large Sparse Reconstruction Model to study how scaling transformer context windows impacts feed-forward 3D reconstruction.<n>We show that expanding the context window -- by substantially increasing the number of active object and image tokens -- remarkably narrows this gap and enables high-fidelity 3D object reconstruction and inverse rendering.
  arXiv  Detail & Related papers  (2026-04-06T21:21:12Z)
• LoST: Level of Semantics Tokenization for 3D Shapes [50.847769883816085]
  State-of-the-art methods rely on geometric level-of-detail (LoD) hierarchies, originally designed for rendering and compression.<n>We propose Level-of-Semantics Tokenization (LoST), which orders salience, such that early prefixes decode complete, plausible shapes.<n>LoST achieves SOTA reconstruction, surpassing previous LoD-based 3D shape tokenizers by large margins on both geometric and semantic reconstruction metrics.
  arXiv  Detail & Related papers  (2026-03-18T17:56:06Z)
• GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction [52.04103235260539]
  We present a diffusion model approach based on Gaussian Splatting representation for 3D object reconstruction from a single view. The model learns to generate 3D objects represented by sets of GS ellipsoids. The final reconstructed objects explicitly come with high-quality 3D structure and texture, and can be efficiently rendered in arbitrary views.
  arXiv  Detail & Related papers  (2024-07-05T03:43:08Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• Semi-supervised Learning from Street-View Images and OpenStreetMap for Automatic Building Height Estimation [59.6553058160943]
  We propose a semi-supervised learning (SSL) method of automatically estimating building height from Mapillary SVI and OpenStreetMap data. The proposed method leads to a clear performance boosting in estimating building heights with a Mean Absolute Error (MAE) around 2.1 meters. The preliminary result is promising and motivates our future work in scaling up the proposed method based on low-cost VGI data.
  arXiv  Detail & Related papers  (2023-07-05T18:16:30Z)
• Scan2LoD3: Reconstructing semantic 3D building models at LoD3 using ray casting and Bayesian networks [40.7734793392562]
  Reconstructing semantic 3D building models at the level of detail (LoD) 3 is a long-standing challenge. We present a novel method, called Scan2LoD3, that accurately reconstructs semantic LoD3 building models. We believe our method can foster the development of probability-driven semantic 3D reconstruction at LoD3.
  arXiv  Detail & Related papers  (2023-05-10T17:01:18Z)
• Combining visibility analysis and deep learning for refinement of semantic 3D building models by conflict classification [3.2662392450935416]
  We propose a method of combining visibility analysis and neural networks for enriching 3D models with window and door features. In the method, occupancy voxels are fused with classified point clouds, which provides semantics to voxels. The semantic voxels and conflicts are combined in a Bayesian network to classify and delineate faccade openings, which are reconstructed using a 3D model library.
  arXiv  Detail & Related papers  (2023-03-10T16:01:30Z)
• Elevation Estimation-Driven Building 3D Reconstruction from Single-View Remote Sensing Imagery [20.001807614214922]
  Building 3D reconstruction from remote sensing images has a wide range of applications in smart cities, photogrammetry and other fields. We propose an efficient DSM estimation-driven reconstruction framework (Building3D) to reconstruct 3D building models from the input single-view remote sensing image. Our Building3D is rooted in the SFFDE network for building elevation prediction, synchronized with a building extraction network for building masks, and then sequentially performs point cloud reconstruction, surface reconstruction (or CityGML model reconstruction)
  arXiv  Detail & Related papers  (2023-01-11T17:20:30Z)
• Translational Symmetry-Aware Facade Parsing for 3D Building Reconstruction [11.263458202880038]
  In this paper, we present a novel translational symmetry-based approach to improving the deep neural networks. We propose a novel scheme to fuse anchor-free detection in a single stage network, which enables the efficient training and better convergence. We employ an off-the-shelf rendering engine like Blender to reconstruct the realistic high-quality 3D models using procedural modeling.
  arXiv  Detail & Related papers  (2021-06-02T03:10:51Z)
• SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion [86.77318031029404]
  We propose a framework that performs scene reconstruction and semantic scene completion jointly in an incremental and real-time manner. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model.
  arXiv  Detail & Related papers  (2020-10-26T15:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.