sat2pc: Estimating Point Cloud of Building Roofs from 2D Satellite Images

• URL: http://arxiv.org/abs/2205.12464v1
• Date: Wed, 25 May 2022 03:24:40 GMT
• Title: sat2pc: Estimating Point Cloud of Building Roofs from 2D Satellite Images
• Authors: Yoones Rezaei, Stephen Lee
• Abstract summary: We propose sat2pc, a deep learning architecture that predicts the point of a building roof from a single 2D satellite image. Our results show that sat2pc was able to outperform existing baselines by at least 18.6%.
• Score: 1.8884278918443564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Three-dimensional (3D) urban models have gained interest because of their applications in many use-cases such as urban planning and virtual reality. However, generating these 3D representations requires LiDAR data, which are not always readily available. Thus, the applicability of automated 3D model generation algorithms is limited to a few locations. In this paper, we propose sat2pc, a deep learning architecture that predicts the point cloud of a building roof from a single 2D satellite image. Our architecture combines Chamfer distance and EMD loss, resulting in better 2D to 3D performance. We extensively evaluate our model and perform ablation studies on a building roof dataset. Our results show that sat2pc was able to outperform existing baselines by at least 18.6%. Further, we show that the predicted point cloud captures more detail and geometric characteristics than other baselines.

Related papers

• Leveraging Large-Scale Pretrained Vision Foundation Models for Label-Efficient 3D Point Cloud Segmentation [67.07112533415116]
  We present a novel framework that adapts various foundational models for the 3D point cloud segmentation task. Our approach involves making initial predictions of 2D semantic masks using different large vision models. To generate robust 3D semantic pseudo labels, we introduce a semantic label fusion strategy that effectively combines all the results via voting.
  arXiv  Detail & Related papers  (2023-11-03T15:41:15Z)
• Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds [4.38301148531795]
  Existing datasets for 3D modeling mainly focus on common objects such as furniture or cars. We present a urban-scale dataset consisting of more than 160 thousands buildings along with corresponding point clouds, mesh and wire-frame models, covering 16 cities in Estonia about 998 Km2. Experimental results indicate that Building3D has challenges of high intra-class variance, data imbalance and large-scale noises.
  arXiv  Detail & Related papers  (2023-07-21T21:38:57Z)
• 3D detection of roof sections from a single satellite image and application to LOD2-building reconstruction [12.693545159861857]
  We propose a method for urban 3D reconstruction named KIBS(textitKeypoints Inference By), which comprises two novel features. We demonstrate the potential of the KIBS method by reconstructing different urban areas in a few minutes, with a Jaccard index for the 2D segmentation of individual roof sections of $88.55%$ and $75.21%$ on our two data sets resp., and a height's mean error of such correctly segmented pixels for the 3D reconstruction of $1.60$ m and $2.06 m on our two data sets resp., hence within the LOD
  arXiv  Detail & Related papers  (2023-07-11T16:23:19Z)
• PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Clouds [29.15589024703907]
  In this paper, we revisit the local point aggregators from the perspective of allocating computational resources. We find that the simplest pillar based models perform surprisingly well considering both accuracy and latency. Our results challenge the common intuition that the detailed geometry modeling is essential to achieve high performance for 3D object detection.
  arXiv  Detail & Related papers  (2023-05-08T17:59:14Z)
• RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects [68.85305626324694]
  Ray-marching in Camera Space (RiCS) is a new method to represent the self-occlusions of foreground objects in 3D into a 2D self-occlusion map. We show that our representation map not only allows us to enhance the image quality but also to model temporally coherent complex shadow effects.
  arXiv  Detail & Related papers  (2022-05-14T05:35:35Z)
• Simple and Effective Synthesis of Indoor 3D Scenes [78.95697556834536]
  We study the problem of immersive 3D indoor scenes from one or more images. Our aim is to generate high-resolution images and videos from novel viewpoints. We propose an image-to-image GAN that maps directly from reprojections of incomplete point clouds to full high-resolution RGB-D images.
  arXiv  Detail & Related papers  (2022-04-06T17:54:46Z)
• ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields [20.00737387884824]
  ImpliCity is a neural representation of the 3D scene as an implicit, continuous occupancy field, driven by learned embeddings of the point cloud and a stereo pair of ortho-photos. With image resolution 0.5$,$m, ImpliCity reaches a median height error of $approx,$0.7$,$m and outperforms competing methods.
  arXiv  Detail & Related papers  (2022-01-24T21:40:16Z)
• SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds [52.624157840253204]
  We introduce SensatUrban, an urban-scale UAV photogrammetry point cloud dataset consisting of nearly three billion points collected from three UK cities, covering 7.6 km2. Each point in the dataset has been labelled with fine-grained semantic annotations, resulting in a dataset that is three times the size of the previous existing largest photogrammetric point cloud dataset.
  arXiv  Detail & Related papers  (2022-01-12T14:48:11Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception [122.53774221136193]
  State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution. A natural remedy is to utilize the 3D voxelization and 3D convolution network. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
  arXiv  Detail & Related papers  (2021-09-12T06:25:11Z)
• Semantic Segmentation on Swiss3DCities: A Benchmark Study on Aerial Photogrammetric 3D Pointcloud Dataset [67.44497676652173]
  We introduce a new outdoor urban 3D pointcloud dataset, covering a total area of 2.7 $km2$, sampled from three Swiss cities. The dataset is manually annotated for semantic segmentation with per-point labels, and is built using photogrammetry from images acquired by multirotors equipped with high-resolution cameras.
  arXiv  Detail & Related papers  (2020-12-23T21:48:47Z)
• A Nearest Neighbor Network to Extract Digital Terrain Models from 3D Point Clouds [1.6249267147413524]
  We present an algorithm that operates on 3D-point clouds and estimates the underlying DTM for the scene using an end-to-end approach. Our model learns neighborhood information and seamlessly integrates this with point-wise and block-wise global features.
  arXiv  Detail & Related papers  (2020-05-21T15:54:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.