ODE-CNN: Omnidirectional Depth Extension Networks

• URL: http://arxiv.org/abs/2007.01475v1
• Date: Fri, 3 Jul 2020 03:14:09 GMT
• Title: ODE-CNN: Omnidirectional Depth Extension Networks
• Authors: Xinjing Cheng, Peng Wang, Yanqi Zhou, Chenye Guan and Ruigang Yang
• Abstract summary: We propose a low-cost 3D sensing system that combines an omnidirectional camera with a calibrated projective depth camera. To accurately recover the missing depths, we design an omnidirectional depth extension convolutional neural network. ODE-CNN significantly outperforms (relatively 33% reduction in-depth error) other state-of-the-art (SoTA) methods.
• Score: 43.40308168978984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Omnidirectional 360{\deg} camera proliferates rapidly for autonomous robots since it significantly enhances the perception ability by widening the field of view(FoV). However, corresponding 360{\deg} depth sensors, which are also critical for the perception system, are still difficult or expensive to have. In this paper, we propose a low-cost 3D sensing system that combines an omnidirectional camera with a calibrated projective depth camera, where the depth from the limited FoV can be automatically extended to the rest of the recorded omnidirectional image. To accurately recover the missing depths, we design an omnidirectional depth extension convolutional neural network(ODE-CNN), in which a spherical feature transform layer(SFTL) is embedded at the end of feature encoding layers, and a deformable convolutional spatial propagation network(D-CSPN) is appended at the end of feature decoding layers. The former resamples the neighborhood of each pixel in the omnidirectional coordination to the projective coordination, which reduces the difficulty of feature learning, and the later automatically finds a proper context to well align the structures in the estimated depths via CNN w.r.t. the reference image, which significantly improves the visual quality. Finally, we demonstrate the effectiveness of proposed ODE-CNN over the popular 360D dataset and show that ODE-CNN significantly outperforms (relatively 33% reduction in-depth error) other state-of-the-art (SoTA) methods.

Related papers

• Pixel-Aligned Multi-View Generation with Depth Guided Decoder [86.1813201212539]
  We propose a novel method for pixel-level image-to-multi-view generation. Unlike prior work, we incorporate attention layers across multi-view images in the VAE decoder of a latent video diffusion model. Our model enables better pixel alignment across multi-view images.
  arXiv  Detail & Related papers  (2024-08-26T04:56:41Z)
• SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning [17.99904937160487]
  We introduce SCIPaD, a novel approach that incorporates spatial clues for unsupervised depth-pose joint learning. SCIPaD achieves a reduction of 22.2% in average translation error and 34.8% in average angular error for camera pose estimation task on the KITTI Odometry dataset.
  arXiv  Detail & Related papers  (2024-07-07T06:52:51Z)
• Neural Contourlet Network for Monocular 360 Depth Estimation [37.82642960470551]
  We provide a new perspective that constructs an interpretable and sparse representation for a 360 image. We propose a neural contourlet network consisting of a convolutional neural network and a contourlet transform branch. In the encoder stage, we design a spatial-spectral fusion module to effectively fuse two types of cues.
  arXiv  Detail & Related papers  (2022-08-03T02:25:55Z)
• HiMODE: A Hybrid Monocular Omnidirectional Depth Estimation Model [3.5290359800552946]
  HiMODE is a novel monocular omnidirectional depth estimation model based on a CNN+Transformer architecture. We show that HiMODE can achieve state-of-the-art performance for 360deg monocular depth estimation.
  arXiv  Detail & Related papers  (2022-04-11T11:11:43Z)
• Unsupervised Depth Completion with Calibrated Backprojection Layers [79.35651668390496]
  We propose a deep neural network architecture to infer dense depth from an image and a sparse point cloud. It is trained using a video stream and corresponding synchronized sparse point cloud, as obtained from a LIDAR or other range sensor, along with the intrinsic calibration parameters of the camera. At inference time, the calibration of the camera, which can be different from the one used for training, is fed as an input to the network along with the sparse point cloud and a single image.
  arXiv  Detail & Related papers  (2021-08-24T05:41:59Z)
• PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation with Neural Positional Encoding and Distilled Matting Loss [49.66736599668501]
  We propose a self-supervised single-view pixel-level accurate depth estimation network, called PLADE-Net. Our method shows unprecedented accuracy levels, exceeding 95% in terms of the $delta1$ metric on the KITTI dataset.
  arXiv  Detail & Related papers  (2021-03-12T15:54:46Z)
• PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective Crop Layers [111.55817466296402]
  We introduce Perspective Crop Layers (PCLs) - a form of perspective crop of the region of interest based on the camera geometry. PCLs deterministically remove the location-dependent perspective effects while leaving end-to-end training and the number of parameters of the underlying neural network. PCL offers an easy way to improve the accuracy of existing 3D reconstruction networks by making them geometry aware.
  arXiv  Detail & Related papers  (2020-11-27T08:48:43Z)
• OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems [88.41004332322788]
  We present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation. We integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency.
  arXiv  Detail & Related papers  (2020-03-18T05:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.