Single Image Depth Prediction Made Better: A Multivariate Gaussian Take

• URL: http://arxiv.org/abs/2303.18164v2
• Date: Tue, 18 Apr 2023 08:52:18 GMT
• Title: Single Image Depth Prediction Made Better: A Multivariate Gaussian Take
• Authors: Ce Liu, Suryansh Kumar, Shuhang Gu, Radu Timofte, Luc Van Gool
• Abstract summary: We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
• Score: 163.14849753700682
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural-network-based single image depth prediction (SIDP) is a challenging task where the goal is to predict the scene's per-pixel depth at test time. Since the problem, by definition, is ill-posed, the fundamental goal is to come up with an approach that can reliably model the scene depth from a set of training examples. In the pursuit of perfect depth estimation, most existing state-of-the-art learning techniques predict a single scalar depth value per-pixel. Yet, it is well-known that the trained model has accuracy limits and can predict imprecise depth. Therefore, an SIDP approach must be mindful of the expected depth variations in the model's prediction at test time. Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution. To this end, we model per-pixel scene depth using a multivariate Gaussian distribution. Moreover, contrary to the existing uncertainty modeling methods -- in the same spirit, where per-pixel depth is assumed to be independent, we introduce per-pixel covariance modeling that encodes its depth dependency w.r.t all the scene points. Unfortunately, per-pixel depth covariance modeling leads to a computationally expensive continuous loss function, which we solve efficiently using the learned low-rank approximation of the overall covariance matrix. Notably, when tested on benchmark datasets such as KITTI, NYU, and SUN-RGB-D, the SIDP model obtained by optimizing our loss function shows state-of-the-art results. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.

Related papers

• Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images [65.41966114373373]
  We present an improved solution to the neural image-based rendering problem in computer vision. The proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time.
  arXiv  Detail & Related papers  (2023-11-08T08:18:23Z)
• Gradient-based Uncertainty for Monocular Depth Estimation [5.7575052885308455]
  In monocular depth estimation, disturbances in the image context, like moving objects or reflecting materials, can easily lead to erroneous predictions. We propose a post hoc uncertainty estimation approach for an already trained and thus fixed depth estimation model. Our approach achieves state-of-the-art uncertainty estimation results on the KITTI and NYU Depth V2 benchmarks without the need to retrain the neural network.
  arXiv  Detail & Related papers  (2022-08-03T12:21:02Z)
• RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation [27.679479140943503]
  We propose a resolution adaptive self-supervised monocular depth estimation method (RA-Depth) by learning the scale invariance of the scene depth. RA-Depth achieves state-of-the-art performance, and also exhibits a good ability of resolution adaptation.
  arXiv  Detail & Related papers  (2022-07-25T08:49:59Z)
• End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotation [10.896567381206715]
  We propose a novel end-to-end learning approach for depth from diffracted rotation. Our approach requires a significantly less complex model and less training data, yet it is superior to existing methods in the task of monocular depth estimation.
  arXiv  Detail & Related papers  (2022-04-14T16:14:37Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)
• PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation with Neural Positional Encoding and Distilled Matting Loss [49.66736599668501]
  We propose a self-supervised single-view pixel-level accurate depth estimation network, called PLADE-Net. Our method shows unprecedented accuracy levels, exceeding 95% in terms of the $delta1$ metric on the KITTI dataset.
  arXiv  Detail & Related papers  (2021-03-12T15:54:46Z)
• Learning Accurate Dense Correspondences and When to Trust Them [161.76275845530964]
  We aim to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. Our approach obtains state-of-the-art results on challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-01-05T18:54:11Z)
• Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration [77.1056200937214]
  We study the formation of the DP pair which links the blur and the depth information. We propose an end-to-end DDDNet (DP-based Depth and De Network) to jointly estimate the depth and restore the image.
  arXiv  Detail & Related papers  (2020-12-01T06:53:57Z)
• Variational Monocular Depth Estimation for Reliability Prediction [12.951621755732544]
  Self-supervised learning for monocular depth estimation is widely investigated as an alternative to supervised learning approach. Previous works have successfully improved the accuracy of depth estimation by modifying the model structure. In this paper, we theoretically formulate a variational model for the monocular depth estimation to predict the reliability of the estimated depth image.
  arXiv  Detail & Related papers  (2020-11-24T06:23:51Z)
• AcED: Accurate and Edge-consistent Monocular Depth Estimation [0.0]
  Single image depth estimation is a challenging problem. We formulate a fully differentiable ordinal regression and train the network in end-to-end fashion. A novel per-pixel confidence map computation for depth refinement is also proposed.
  arXiv  Detail & Related papers  (2020-06-16T15:21:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.