SphereDepth: Panorama Depth Estimation from Spherical Domain

• URL: http://arxiv.org/abs/2208.13714v2
• Date: Tue, 30 Aug 2022 03:01:52 GMT
• Title: SphereDepth: Panorama Depth Estimation from Spherical Domain
• Authors: Qingsong Yan, Qiang Wang, Kaiyong Zhao, Bo Li, Xiaowen Chu, Fei Deng
• Abstract summary: This paper proposes SphereDepth, a novel panorama depth estimation method. It predicts the depth directly on the spherical mesh without projection preprocessing. It achieves comparable results with the state-of-the-art methods of panorama depth estimation.
• Score: 17.98608948955211
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The panorama image can simultaneously demonstrate complete information of the surrounding environment and has many advantages in virtual tourism, games, robotics, etc. However, the progress of panorama depth estimation cannot completely solve the problems of distortion and discontinuity caused by the commonly used projection methods. This paper proposes SphereDepth, a novel panorama depth estimation method that predicts the depth directly on the spherical mesh without projection preprocessing. The core idea is to establish the relationship between the panorama image and the spherical mesh and then use a deep neural network to extract features on the spherical domain to predict depth. To address the efficiency challenges brought by the high-resolution panorama data, we introduce two hyper-parameters for the proposed spherical mesh processing framework to balance the inference speed and accuracy. Validated on three public panorama datasets, SphereDepth achieves comparable results with the state-of-the-art methods of panorama depth estimation. Benefiting from the spherical domain setting, SphereDepth can generate a high-quality point cloud and significantly alleviate the issues of distortion and discontinuity.

Related papers

• Robust and Flexible Omnidirectional Depth Estimation with Multiple 360° Cameras [8.850391039025077]
  We use geometric constraints and redundant information of multiple 360-degree cameras to achieve robust and flexible omnidirectional depth estimation. Our two algorithms achieve state-of-the-art performance, accurately predicting depth maps even when provided with soiled panorama inputs.
  arXiv  Detail & Related papers  (2024-09-23T07:31:48Z)
• SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model [63.685132323224124]
  Controllable spherical panoramic image generation holds substantial applicative potential across a variety of domains. In this paper, we introduce a novel framework of SphereDiffusion to address these unique challenges. Experiments on Structured3D dataset show that SphereDiffusion significantly improves the quality of controllable spherical image generation and relatively reduces around 35% FID on average.
  arXiv  Detail & Related papers  (2024-03-15T06:26:46Z)
• GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection [95.8940731298518]
  We propose a novel Geometry Uncertainty Propagation Network (GUPNet++) It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning. Experiments show that the proposed approach not only obtains (state-of-the-art) SOTA performance in image-based monocular 3D detection but also demonstrates superiority in efficacy with a simplified framework.
  arXiv  Detail & Related papers  (2023-10-24T08:45:15Z)
• Calibrating Panoramic Depth Estimation for Practical Localization and Mapping [20.621442016969976]
  The absolute depth values of surrounding environments provide crucial cues for various assistive technologies, such as localization, navigation, and 3D structure estimation. We propose that accurate depth estimated from panoramic images can serve as a powerful and light-weight input for a wide range of downstream tasks requiring 3D information.
  arXiv  Detail & Related papers  (2023-08-27T04:50:05Z)
• PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas [54.4948540627471]
  We propose PanoGRF, Generalizable Spherical Radiance Fields for Wide-baseline Panoramas. Unlike generalizable radiance fields trained on perspective images, PanoGRF avoids the information loss from panorama-to-perspective conversion. Results on multiple panoramic datasets demonstrate that PanoGRF significantly outperforms state-of-the-art generalizable view synthesis methods.
  arXiv  Detail & Related papers  (2023-06-02T13:35:07Z)
• ${S}^{2}$Net: Accurate Panorama Depth Estimation on Spherical Surface [4.649656275858966]
  We propose an end-to-end deep network for monocular panorama depth estimation on a unit spherical surface. Specifically, we project the feature maps extracted from equirectangular images onto unit spherical surface sampled by uniformly distributed grids. We propose a global cross-attention-based fusion module to fuse the feature maps from skip connection and enhance the ability to obtain global context.
  arXiv  Detail & Related papers  (2023-01-14T07:39:15Z)
• PanoFormer: Panorama Transformer for Indoor 360{\deg} Depth Estimation [35.698249161263966]
  Existing panoramic depth estimation methods based on convolutional neural networks (CNNs) focus on removing panoramic distortions. This paper proposes the panorama transformer (named PanoFormer) to estimate the depth in panorama images. In particular, we divide patches on the spherical tangent domain into tokens to reduce the negative effect of panoramic distortions.
  arXiv  Detail & Related papers  (2022-03-17T12:19:43Z)
• A learning-based view extrapolation method for axial super-resolution [52.748944517480155]
  Axial light field resolution refers to the ability to distinguish features at different depths by refocusing. We propose a learning-based method to extrapolate novel views from axial volumes of sheared epipolar plane images.
  arXiv  Detail & Related papers  (2021-03-11T07:22:13Z)
• Depth Completion using Piecewise Planar Model [94.0808155168311]
  A depth map can be represented by a set of learned bases and can be efficiently solved in a closed form solution. However, one issue with this method is that it may create artifacts when colour boundaries are inconsistent with depth boundaries. We enforce a more strict model in depth recovery: a piece-wise planar model.
  arXiv  Detail & Related papers  (2020-12-06T07:11:46Z)
• Distortion-aware Monocular Depth Estimation for Omnidirectional Images [26.027353545874522]
  We propose a Distortion-Aware Monocular Omnidirectional (DAMO) dense depth estimation network to address this challenge on indoor panoramas. First, we introduce a distortion-aware module to extract calibrated semantic features from omnidirectional images. Second, we introduce a plug-and-play spherical-aware weight matrix for our objective function to handle the uneven distribution of areas projected from a sphere.
  arXiv  Detail & Related papers  (2020-10-18T08:47:57Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.