3D reconstruction from spherical images: A review of techniques, applications, and prospects

• URL: http://arxiv.org/abs/2302.04495v4
• Date: Thu, 18 May 2023 01:23:40 GMT
• Title: 3D reconstruction from spherical images: A review of techniques, applications, and prospects
• Authors: San Jiang, Yaxin Li, Duojie Weng, Kan You, Wu Chen
• Abstract summary: 3D reconstruction plays an increasingly important role in modern photogrammetric systems. With the rapid evolution and extensive use of professional and consumer-grade spherical cameras, spherical images show great potential for the 3D modeling of urban and indoor scenes. This research provides a thorough survey of the state-of-the-art for 3D reconstruction of spherical images in terms of data acquisition, feature detection and matching, image orientation, and dense matching.
• Score: 2.6432771146480283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D reconstruction plays an increasingly important role in modern photogrammetric systems. Conventional satellite or aerial-based remote sensing (RS) platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities. Even with low-altitude UAVs (Unmanned Aerial Vehicles), 3D reconstruction in complicated situations, such as urban canyons and indoor scenes, is challenging due to frequent tracking failures between camera frames and high data collection costs. Recently, spherical images have been extensively used due to the capability of recording surrounding environments from one camera exposure. In contrast to perspective images with limited FOV (Field of View), spherical images can cover the whole scene with full horizontal and vertical FOV and facilitate camera tracking and data acquisition in these complex scenes. With the rapid evolution and extensive use of professional and consumer-grade spherical cameras, spherical images show great potential for the 3D modeling of urban and indoor scenes. Classical 3D reconstruction pipelines, however, cannot be directly used for spherical images. Besides, there exist few software packages that are designed for the 3D reconstruction of spherical images. As a result, this research provides a thorough survey of the state-of-the-art for 3D reconstruction of spherical images in terms of data acquisition, feature detection and matching, image orientation, and dense matching as well as presenting promising applications and discussing potential prospects. We anticipate that this study offers insightful clues to direct future research.

Related papers

• Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos [76.07894127235058]
  We present a system for mining high-quality 4D reconstructions from internet stereoscopic, wide-angle videos. We use this method to generate large-scale data in the form of world-consistent, pseudo-metric 3D point clouds. We demonstrate the utility of this data by training a variant of DUSt3R to predict structure and 3D motion from real-world image pairs.
  arXiv  Detail & Related papers  (2024-12-12T18:59:54Z)
• Reconstructing Satellites in 3D from Amateur Telescope Images [42.850623200702394]
  This paper proposes a framework for the 3D reconstruction of satellites in low-Earth orbit, utilizing videos captured by small amateur telescopes. The video data obtained from these telescopes differ significantly from data for standard 3D reconstruction tasks, characterized by intense motion blur, atmospheric turbulence, pervasive background light pollution, extended focal length and constrained observational perspectives. We apply a customized Structure from Motion (SfM) approach, followed by an improved 3D Gaussian splatting algorithm, to achieve high-fidelity 3D model reconstruction.
  arXiv  Detail & Related papers  (2024-04-29T03:13:09Z)
• Towards 3D Vision with Low-Cost Single-Photon Cameras [24.711165102559438]
  We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. Our work draws a connection between image-based modeling and active range scanning and is a step towards 3D vision with single-photon cameras.
  arXiv  Detail & Related papers  (2024-03-26T15:40:05Z)
• R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras [106.52409577316389]
  R3D3 is a multi-camera system for dense 3D reconstruction and ego-motion estimation. Our approach exploits spatial-temporal information from multiple cameras, and monocular depth refinement. We show that this design enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor environments.
  arXiv  Detail & Related papers  (2023-08-28T17:13:49Z)
• Multi-Modal Dataset Acquisition for Photometrically Challenging Object [56.30027922063559]
  This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects. We propose a novel annotation and acquisition pipeline that enhances existing 3D perception and 6D object pose datasets.
  arXiv  Detail & Related papers  (2023-08-21T10:38:32Z)
• 3D Reconstruction of Spherical Images based on Incremental Structure from Motion [2.6432771146480283]
  This study investigates the algorithms for the relative orientation using spherical correspondences, absolute orientation using 3D correspondences between scene and spherical points, and the cost functions for BA (bundle adjustment) optimization. An incremental SfM (Structure from Motion) workflow has been proposed for spherical images using the above-mentioned algorithms.
  arXiv  Detail & Related papers  (2023-06-22T09:49:28Z)
• State of the Art in Dense Monocular Non-Rigid 3D Reconstruction [100.9586977875698]
  3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views.
  arXiv  Detail & Related papers  (2022-10-27T17:59:53Z)
• Aerial Monocular 3D Object Detection [67.20369963664314]
  DVDET is proposed to achieve aerial monocular 3D object detection in both the 2D image space and the 3D physical space. To address the severe view deformation issue, we propose a novel trainable geo-deformable transformation module. To encourage more researchers to investigate this area, we will release the dataset and related code.
  arXiv  Detail & Related papers  (2022-08-08T08:32:56Z)
• Towards Non-Line-of-Sight Photography [48.491977359971855]
  Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects. Active NLOS imaging systems rely on the capture of the time of flight of light through the scene. We propose a new problem formulation, called NLOS photography, to specifically address this deficiency.
  arXiv  Detail & Related papers  (2021-09-16T08:07:13Z)
• 3D Reconstruction through Fusion of Cross-View Images [4.644618399001]
  3D recovery from multi-stereo and stereo images serves many applications in computer vision, remote sensing and Geomatics. We introduce our framework that takes ground-view images and satellite images for full 3D recovery. We demonstrate our proposed framework on a dataset consisting of twelve satellite images and 150k video frames acquired through a vehicle-mounted Go-pro camera.
  arXiv  Detail & Related papers  (2021-06-27T18:31:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.