3D reconstruction from spherical images: A review of techniques, applications, and prospects

• URL: http://arxiv.org/abs/2302.04495v4
• Date: Thu, 18 May 2023 01:23:40 GMT
• Title: 3D reconstruction from spherical images: A review of techniques, applications, and prospects
• Authors: San Jiang, Yaxin Li, Duojie Weng, Kan You, Wu Chen
• Abstract summary: 3D reconstruction plays an increasingly important role in modern photogrammetric systems. With the rapid evolution and extensive use of professional and consumer-grade spherical cameras, spherical images show great potential for the 3D modeling of urban and indoor scenes. This research provides a thorough survey of the state-of-the-art for 3D reconstruction of spherical images in terms of data acquisition, feature detection and matching, image orientation, and dense matching.
• Score: 2.6432771146480283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D reconstruction plays an increasingly important role in modern photogrammetric systems. Conventional satellite or aerial-based remote sensing (RS) platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities. Even with low-altitude UAVs (Unmanned Aerial Vehicles), 3D reconstruction in complicated situations, such as urban canyons and indoor scenes, is challenging due to frequent tracking failures between camera frames and high data collection costs. Recently, spherical images have been extensively used due to the capability of recording surrounding environments from one camera exposure. In contrast to perspective images with limited FOV (Field of View), spherical images can cover the whole scene with full horizontal and vertical FOV and facilitate camera tracking and data acquisition in these complex scenes. With the rapid evolution and extensive use of professional and consumer-grade spherical cameras, spherical images show great potential for the 3D modeling of urban and indoor scenes. Classical 3D reconstruction pipelines, however, cannot be directly used for spherical images. Besides, there exist few software packages that are designed for the 3D reconstruction of spherical images. As a result, this research provides a thorough survey of the state-of-the-art for 3D reconstruction of spherical images in terms of data acquisition, feature detection and matching, image orientation, and dense matching as well as presenting promising applications and discussing potential prospects. We anticipate that this study offers insightful clues to direct future research.

Related papers

• Reconstructing Satellites in 3D from Amateur Telescope Images [44.20773507571372]
  This paper proposes a framework for the 3D reconstruction of satellites in low-Earth orbit, utilizing videos captured by small amateur telescopes. The video data obtained from these telescopes differ significantly from data for standard 3D reconstruction tasks, characterized by intense motion blur, atmospheric turbulence, pervasive background light pollution, extended focal length and constrained observational perspectives. We validate our approach using both synthetic datasets and actual observations of China's Space Station, showcasing its significant advantages over existing methods in reconstructing 3D space objects from ground-based observations.
  arXiv  Detail & Related papers  (2024-04-29T03:13:09Z)
• Towards 3D Vision with Low-Cost Single-Photon Cameras [24.711165102559438]
  We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. Our work draws a connection between image-based modeling and active range scanning and is a step towards 3D vision with single-photon cameras.
  arXiv  Detail & Related papers  (2024-03-26T15:40:05Z)
• ReconFusion: 3D Reconstruction with Diffusion Priors [104.73604630145847]
  We present ReconFusion to reconstruct real-world scenes using only a few photos. Our approach leverages a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets. Our method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions.
  arXiv  Detail & Related papers  (2023-12-05T18:59:58Z)
• R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras [106.52409577316389]
  R3D3 is a multi-camera system for dense 3D reconstruction and ego-motion estimation. Our approach exploits spatial-temporal information from multiple cameras, and monocular depth refinement. We show that this design enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor environments.
  arXiv  Detail & Related papers  (2023-08-28T17:13:49Z)
• Multi-Modal Dataset Acquisition for Photometrically Challenging Object [56.30027922063559]
  This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects. We propose a novel annotation and acquisition pipeline that enhances existing 3D perception and 6D object pose datasets.
  arXiv  Detail & Related papers  (2023-08-21T10:38:32Z)
• 3D Reconstruction of Spherical Images based on Incremental Structure from Motion [2.6432771146480283]
  This study investigates the algorithms for the relative orientation using spherical correspondences, absolute orientation using 3D correspondences between scene and spherical points, and the cost functions for BA (bundle adjustment) optimization. An incremental SfM (Structure from Motion) workflow has been proposed for spherical images using the above-mentioned algorithms.
  arXiv  Detail & Related papers  (2023-06-22T09:49:28Z)
• 3D Data Augmentation for Driving Scenes on Camera [50.41413053812315]
  We propose a 3D data augmentation approach termed Drive-3DAug, aiming at augmenting the driving scenes on camera in the 3D space. We first utilize Neural Radiance Field (NeRF) to reconstruct the 3D models of background and foreground objects. Then, augmented driving scenes can be obtained by placing the 3D objects with adapted location and orientation at the pre-defined valid region of backgrounds.
  arXiv  Detail & Related papers  (2023-03-18T05:51:05Z)
• State of the Art in Dense Monocular Non-Rigid 3D Reconstruction [100.9586977875698]
  3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views.
  arXiv  Detail & Related papers  (2022-10-27T17:59:53Z)
• Towards Non-Line-of-Sight Photography [48.491977359971855]
  Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects. Active NLOS imaging systems rely on the capture of the time of flight of light through the scene. We propose a new problem formulation, called NLOS photography, to specifically address this deficiency.
  arXiv  Detail & Related papers  (2021-09-16T08:07:13Z)
• 3D Reconstruction through Fusion of Cross-View Images [4.644618399001]
  3D recovery from multi-stereo and stereo images serves many applications in computer vision, remote sensing and Geomatics. We introduce our framework that takes ground-view images and satellite images for full 3D recovery. We demonstrate our proposed framework on a dataset consisting of twelve satellite images and 150k video frames acquired through a vehicle-mounted Go-pro camera.
  arXiv  Detail & Related papers  (2021-06-27T18:31:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.