Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery

• URL: http://arxiv.org/abs/2605.26370v1
• Date: Mon, 25 May 2026 22:37:11 GMT
• Title: Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery
• Abstract summary: We present a method for jointly predicting instance-level roof segment masks from a single aerial orthophoto.<n>We introduce two key innovations: a conditional azimuth loss that suppresses supervision for flat roof segments, and a log-normalized height representation.<n>Our method achieves a mean absolute error of approximately 4 degrees for roof slope, 7 degrees for azimuth, and 1 meter for building height, with an instance segmentation AP$_50$ of 0.566.
• Score: 18.60580673640467
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a method for jointly predicting instance-level roof segment masks together with three continuous geometric attributes -- building height, roof slope, and roof azimuth -- from a single aerial orthophoto. Our approach extends Mask R-CNN with a dedicated attribute regression branch and introduces two key innovations: a conditional azimuth loss that suppresses supervision for flat roof segments where azimuth labels are inherently noisy, and a log-normalized height representation that addresses the heavily skewed distribution of building heights. We train and evaluate on a large-scale dataset of Dutch aerial images paired with automatically derived ground truth from 3DBAG, a nationwide LiDAR-based 3D building dataset. Using a DINOv3 ConvNeXt-Base backbone, our method achieves a mean absolute error of approximately 4 degrees for roof slope, 7 degrees for azimuth, and 1 meter for building height, with an instance segmentation AP$_{50}$ of 0.566. The predicted per-segment masks and attributes are sufficient to reconstruct simplified 3D building models (LoD2) from a single overhead image, requiring expensive 3D reference data only for training.

Related papers

• SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models [51.121529170856064]
  We propose SemCityLoc, a semantic-geometric alignment system that reframes aerial pose estimation.<n>Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models.<n>Experiments demonstrate substantial improvements over existing map-based approaches.
  arXiv  Detail & Related papers  (2026-06-25T18:13:39Z)
• Multi-Modal Building Inspection via Perceiver IO Fusion of Satellite and Street-Level Imagery [18.60580673640467]
  We present a multi-modal classification framework that fuses satellite and street-level imagery through a Perceiver IO architecture.<n>We construct a large-scale dataset of 32,135 buildings spanning ten countries, pairing satellite images with up to eight street-level views per segment.<n>The Perceiver IO fusion model improves over all other fusion strategies and yields substantial per-class gains for attributes visible from street level.
  arXiv  Detail & Related papers  (2026-05-25T23:01:05Z)
• Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence [78.1406635199656]
  Holi-Spatial is the first fully automated, large-scale, spatially-aware multimodal dataset constructed from raw video inputs without human intervention.<n>Holi-Spatial-4M is the first large-scale, high-quality 3D semantic dataset, containing 12K optimized 3DGS scenes, 1.3M 2D masks, 320K 3D bounding boxes, 320K instance captions, 1.2M 3D grounding instances, and 1.2M spatial QA pairs.
  arXiv  Detail & Related papers  (2026-03-08T14:49:20Z)
• Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video [76.32954467706581]
  We propose SAGE, a framework for Scalable Adaptation of GEometric foundation models from raw video streams.<n>We use a hierarchical mining pipeline to transform videos into training trajectories and hybrid supervision.<n>Experiments show that SAGE significantly enhances zero-shot generalization, reducing Chamfer Distance by 20-42% on unseen benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T09:53:21Z)
• Zero-shot Inexact CAD Model Alignment from a Single Image [53.37898107159792]
  A practical approach to infer 3D scene structure from a single image is to retrieve a closely matching 3D model from a database and align it with the object in the image.<n>Existing methods rely on supervised training with images and pose annotations, which limits them to a narrow set of object categories.<n>We propose a weakly supervised 9-DoF alignment method for inexact 3D models that requires no pose annotations and generalizes to unseen categories.
  arXiv  Detail & Related papers  (2025-07-04T04:46:59Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings [5.093758132026397]
  We propose a novel query-based approach for extracting building outlines from aerial or satellite imagery. We formulate each polygon as a query and constrain the query attention on the most relevant regions of a potential building. We evaluate our method on the vectorized building outline extraction dataset CrowdAI and the 2D floorplan reconstruction dataset Structured3D.
  arXiv  Detail & Related papers  (2024-07-20T16:12:51Z)
• X-3D: Explicit 3D Structure Modeling for Point Cloud Recognition [73.0588783479853]
  X-3D is an explicit 3D structure modeling approach. It captures explicit local structural information within the input 3D space. It produces dynamic kernels with shared weights for all neighborhood points within the current local region.
  arXiv  Detail & Related papers  (2024-04-23T13:15:35Z)
• RoofDiffusion: Constructing Roofs from Severely Corrupted Point Data via Diffusion [0.3277163122167433]
  RoofDiffusion is a new end-to-end self-supervised diffusion technique for robustly completing roof height maps. A variant, No-FP RoofDiffusion, simultaneously predicts building footprints and heights. Tested with the leading City3D algorithm, RoofDiffusion noticeably improves 3D building reconstruction.
  arXiv  Detail & Related papers  (2024-04-14T15:50:10Z)
• 3D detection of roof sections from a single satellite image and application to LOD2-building reconstruction [12.693545159861857]
  We propose a method for urban 3D reconstruction named KIBS(textitKeypoints Inference By), which comprises two novel features. We demonstrate the potential of the KIBS method by reconstructing different urban areas in a few minutes, with a Jaccard index for the 2D segmentation of individual roof sections of $88.55%$ and $75.21%$ on our two data sets resp., and a height's mean error of such correctly segmented pixels for the 3D reconstruction of $1.60$ m and $2.06 m on our two data sets resp., hence within the LOD
  arXiv  Detail & Related papers  (2023-07-11T16:23:19Z)
• Learning to Generate 3D Representations of Building Roofs Using Single-View Aerial Imagery [68.3565370706598]
  We present a novel pipeline for learning the conditional distribution of a building roof mesh given pixels from an aerial image. Unlike alternative methods that require multiple images of the same object, our approach enables estimating 3D roof meshes using only a single image for predictions.
  arXiv  Detail & Related papers  (2023-03-20T15:47:05Z)
• Learning Hyperbolic Representations for Unsupervised 3D Segmentation [3.516233423854171]
  We propose learning effective representations of 3D patches for unsupervised segmentation through a variational autoencoder (VAE) with a hyperbolic latent space and a proposed gyroplane convolutional layer. We demonstrate the effectiveness of our hyperbolic representations for unsupervised 3D segmentation on a hierarchical toy dataset, BraTS whole tumor dataset, and cryogenic electron microscopy data.
  arXiv  Detail & Related papers  (2020-12-03T02:15:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.