Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction

• URL: http://arxiv.org/abs/2002.09085v2
• Date: Thu, 28 May 2020 07:41:15 GMT
• Title: Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction
• Authors: Qing Zhu, Zhendong Wang, Han Hu, Linfu Xie, Xuming Ge, Yeting Zhang
• Abstract summary: Integration of aerial and ground images has been proved as an efficient approach to enhance the surface reconstruction in urban environments. Previous studies based on geometry-aware image rectification have alleviated this problem. We propose a novel approach: leveraging photogrammetric mesh models for aerial-ground image matching.
• Score: 19.551088857830944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integration of aerial and ground images has been proved as an efficient approach to enhance the surface reconstruction in urban environments. However, as the first step, the feature point matching between aerial and ground images is remarkably difficult, due to the large differences in viewpoint and illumination conditions. Previous studies based on geometry-aware image rectification have alleviated this problem, but the performance and convenience of this strategy is limited by several flaws, e.g. quadratic image pairs, segregated extraction of descriptors and occlusions. To address these problems, we propose a novel approach: leveraging photogrammetric mesh models for aerial-ground image matching. The methods of this proposed approach have linear time complexity with regard to the number of images, can explicitly handle low overlap using multi-view images and can be directly injected into off-the-shelf structure-from-motion (SfM) and multi-view stereo (MVS) solutions. First, aerial and ground images are reconstructed separately and initially co-registered through weak georeferencing data. Second, aerial models are rendered to the initial ground views, in which the color, depth and normal images are obtained. Then, the synthesized color images and the corresponding ground images are matched by comparing the descriptors, filtered by local geometrical information, and then propagated to the aerial views using depth images and patch-based matching. Experimental evaluations using various datasets confirm the superior performance of the proposed methods in aerial-ground image matching. In addition, incorporation of the existing SfM and MVS solutions into these methods enables more complete and accurate models to be directly obtained.

Related papers

• RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network [37.759675702107586]
  Predicting accurate maps of objects from two-dimensional images in regions of complex structure spatial material variations is challenging. We propose a method of calibrated feature information from different resolution stages and scales of the image. This approach preserves more physical information, such as texture and geometry of the object in complex regions.
  arXiv  Detail & Related papers  (2024-04-11T14:05:37Z)
• Holistic Inverse Rendering of Complex Facade via Aerial 3D Scanning [38.72679977945778]
  We use multi-view aerial images to reconstruct the geometry, lighting, and material of facades using neural signed distance fields (SDFs) The experiment demonstrates the superior quality of our method on facade holistic inverse rendering, novel view synthesis, and scene editing compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-11-20T15:03:56Z)
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion [105.16622018766236]
  Wonder3D is a novel method for efficiently generating high-fidelity textured meshes from single-view images. To holistically improve the quality, consistency, and efficiency of image-to-3D tasks, we propose a cross-domain diffusion model.
  arXiv  Detail & Related papers  (2023-10-23T15:02:23Z)
• $PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction [97.06927852165464]
  Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision. We propose a novel method for single-image 3D reconstruction which generates a sparse point cloud via a conditional denoising diffusion process.
  arXiv  Detail & Related papers  (2023-02-21T13:37:07Z)
• Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images [15.614631883233898]
  Existing methods have shown promising results leveraging monocular depth estimation and color inpainting with layered depth representations. We propose a new method based on the multiplane image (MPI) representation. The experiments on both synthetic and real datasets demonstrate that our trained model works remarkably well and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2022-05-24T02:57:16Z)
• Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
  We present a modern solution to the multi-view photometric stereo problem (MVPS) We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
  arXiv  Detail & Related papers  (2021-10-11T20:20:03Z)
• Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Images [69.5662419067878]
  Grounding referring expressions in RGBD image has been an emerging field. We present a novel task of 3D visual grounding in single-view RGBD image where the referred objects are often only partially scanned due to occlusion. Our approach first fuses the language and the visual features at the bottom level to generate a heatmap that localizes the relevant regions in the RGBD image. Then our approach conducts an adaptive feature learning based on the heatmap and performs the object-level matching with another visio-linguistic fusion to finally ground the referred object.
  arXiv  Detail & Related papers  (2021-03-14T11:18:50Z)
• Single Image Brightening via Multi-Scale Exposure Fusion with Hybrid Learning [48.890709236564945]
  A small ISO and a small exposure time are usually used to capture an image in the back or low light conditions. In this paper, a single image brightening algorithm is introduced to brighten such an image. The proposed algorithm includes a unique hybrid learning framework to generate two virtual images with large exposure times.
  arXiv  Detail & Related papers  (2020-07-04T08:23:07Z)
• Photometric Multi-View Mesh Refinement for High-Resolution Satellite Images [24.245977127434212]
  State-of-the-art reconstruction methods typically generate 2.5D elevation data. We present an approach to recover full 3D surface meshes from multi-view satellite imagery.
  arXiv  Detail & Related papers  (2020-05-10T20:37:54Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.