Distortion-aware Monocular Depth Estimation for Omnidirectional Images

• URL: http://arxiv.org/abs/2010.08942v2
• Date: Mon, 30 Nov 2020 01:41:15 GMT
• Title: Distortion-aware Monocular Depth Estimation for Omnidirectional Images
• Authors: Hong-Xiang Chen and Kunhong Li and Zhiheng Fu and Mengyi Liu and Zonghao Chen and Yulan Guo
• Abstract summary: We propose a Distortion-Aware Monocular Omnidirectional (DAMO) dense depth estimation network to address this challenge on indoor panoramas. First, we introduce a distortion-aware module to extract calibrated semantic features from omnidirectional images. Second, we introduce a plug-and-play spherical-aware weight matrix for our objective function to handle the uneven distribution of areas projected from a sphere.
• Score: 26.027353545874522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A main challenge for tasks on panorama lies in the distortion of objects among images. In this work, we propose a Distortion-Aware Monocular Omnidirectional (DAMO) dense depth estimation network to address this challenge on indoor panoramas with two steps. First, we introduce a distortion-aware module to extract calibrated semantic features from omnidirectional images. Specifically, we exploit deformable convolution to adjust its sampling grids to geometric variations of distorted objects on panoramas and then utilize a strip pooling module to sample against horizontal distortion introduced by inverse gnomonic projection. Second, we further introduce a plug-and-play spherical-aware weight matrix for our objective function to handle the uneven distribution of areas projected from a sphere. Experiments on the 360D dataset show that the proposed method can effectively extract semantic features from distorted panoramas and alleviate the supervision bias caused by distortion. It achieves state-of-the-art performance on the 360D dataset with high efficiency.

Related papers

• RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation [88.54817424560056]
  We propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification.
  arXiv  Detail & Related papers  (2024-06-27T06:38:56Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception. Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• Distortion-aware Transformer in 360{\deg} Salient Object Detection [44.74647420381127]
  We propose a Transformer-based model called DATFormer to address the distortion problem. To exploit the unique characteristics of 360deg data, we present a learnable relation matrix. Our model outperforms existing 2D SOD (salient object detection) and 360 SOD methods.
  arXiv  Detail & Related papers  (2023-08-07T07:28:24Z)
• Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird's Eye View [44.78243406441798]
  This paper focuses on leveraging geometry information, such as depth, to model such feature transformation. We first lift the 2D image features to the 3D space defined for the ego vehicle via a predicted parametric depth distribution for each pixel in each view. We then aggregate the 3D feature volume based on the 3D space occupancy derived from depth to the BEV frame.
  arXiv  Detail & Related papers  (2023-07-09T06:07:22Z)
• Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
  We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique. First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion. To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
  arXiv  Detail & Related papers  (2023-02-16T10:40:55Z)
• Multi-Projection Fusion and Refinement Network for Salient Object Detection in 360{\deg} Omnidirectional Image [141.10227079090419]
  We propose a Multi-Projection Fusion and Refinement Network (MPFR-Net) to detect the salient objects in 360deg omnidirectional image. MPFR-Net uses the equirectangular projection image and four corresponding cube-unfolding images as inputs. Experimental results on two omnidirectional datasets demonstrate that the proposed approach outperforms the state-of-the-art methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-12-23T14:50:40Z)
• SphereDepth: Panorama Depth Estimation from Spherical Domain [17.98608948955211]
  This paper proposes SphereDepth, a novel panorama depth estimation method. It predicts the depth directly on the spherical mesh without projection preprocessing. It achieves comparable results with the state-of-the-art methods of panorama depth estimation.
  arXiv  Detail & Related papers  (2022-08-29T16:50:19Z)
• Towards Model Generalization for Monocular 3D Object Detection [57.25828870799331]
  We present an effective unified camera-generalized paradigm (CGP) for Mono3D object detection. We also propose the 2D-3D geometry-consistent object scaling strategy (GCOS) to bridge the gap via an instance-level augment. Our method called DGMono3D achieves remarkable performance on all evaluated datasets and surpasses the SoTA unsupervised domain adaptation scheme.
  arXiv  Detail & Related papers  (2022-05-23T23:05:07Z)
• Distortion-Tolerant Monocular Depth Estimation On Omnidirectional Images Using Dual-cubemap [37.82642960470551]
  We propose a distortion-tolerant omnidirectional depth estimation algorithm using a dual-cubemap. In DCDE module, we present a rotation-based dual-cubemap model to estimate the accurate NFoV depth. Then a boundary revision module is designed to smooth the discontinuous boundaries, which contributes to the precise and visually continuous omnidirectional depths.
  arXiv  Detail & Related papers  (2022-03-18T04:20:36Z)
• Wide-angle Image Rectification: A Survey [86.36118799330802]
  wide-angle images contain distortions that violate the assumptions underlying pinhole camera models. Image rectification, which aims to correct these distortions, can solve these problems. We present a detailed description and discussion of the camera models used in different approaches. Next, we review both traditional geometry-based image rectification methods and deep learning-based methods.
  arXiv  Detail & Related papers  (2020-10-30T17:28:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.