MEDeA: Multi-view Efficient Depth Adjustment

• URL: http://arxiv.org/abs/2406.12048v1
• Date: Mon, 17 Jun 2024 19:39:13 GMT
• Title: MEDeA: Multi-view Efficient Depth Adjustment
• Authors: Mikhail Artemyev, Anna Vorontsova, Anna Sokolova, Alexander Limonov,
• Abstract summary: MEDeA is an efficient multi-view test-time depth adjustment method that is an order of magnitude faster than existing test-time approaches. Our method sets a new state-of-the-art on TUM RGB-D, 7Scenes, and ScanNet benchmarks and successfully handles smartphone-captured data from ARKitScenes dataset.
• Score: 45.90423821963144
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The majority of modern single-view depth estimation methods predict relative depth and thus cannot be directly applied in many real-world scenarios, despite impressive performance in the benchmarks. Moreover, single-view approaches cannot guarantee consistency across a sequence of frames. Consistency is typically addressed with test-time optimization of discrepancy across views; however, it takes hours to process a single scene. In this paper, we present MEDeA, an efficient multi-view test-time depth adjustment method, that is an order of magnitude faster than existing test-time approaches. Given RGB frames with camera parameters, MEDeA predicts initial depth maps, adjusts them by optimizing local scaling coefficients, and outputs temporally-consistent depth maps. Contrary to test-time methods requiring normals, optical flow, or semantics estimation, MEDeA produces high-quality predictions with a depth estimation network solely. Our method sets a new state-of-the-art on TUM RGB-D, 7Scenes, and ScanNet benchmarks and successfully handles smartphone-captured data from ARKitScenes dataset.

Related papers

• InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds [91.77050739918037]
  Novel view synthesis (NVS) from a sparse set of images has advanced significantly in 3D computer vision. It relies on precise initial estimation of camera parameters using Structure-from-Motion (SfM) In this study, we introduce a novel and efficient framework to enhance robust NVS from sparse-view images.
  arXiv  Detail & Related papers  (2024-03-29T17:29:58Z)
• Metrically Scaled Monocular Depth Estimation through Sparse Priors for Underwater Robots [0.0]
  We formulate a deep learning model that fuses sparse depth measurements from triangulated features to improve the depth predictions. The network is trained in a supervised fashion on the forward-looking underwater dataset, FLSea. The method achieves real-time performance, running at 160 FPS on a laptop GPU and 7 FPS on a single CPU core.
  arXiv  Detail & Related papers  (2023-10-25T16:32:31Z)
• ARAI-MVSNet: A multi-view stereo depth estimation network with adaptive depth range and depth interval [19.28042366225802]
  Multi-View Stereo(MVS) is a fundamental problem in geometric computer vision. We present a novel multi-stage coarse-to-fine framework to achieve adaptive all-pixel depth range and depth interval. Our model achieves state-of-the-art performance and yields competitive generalization ability.
  arXiv  Detail & Related papers  (2023-08-17T14:52:11Z)
• Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
  We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
  arXiv  Detail & Related papers  (2023-03-31T16:01:03Z)
• VA-DepthNet: A Variational Approach to Single Image Depth Prediction [163.14849753700682]
  VA-DepthNet is a simple, effective, and accurate deep neural network approach for the single-image depth prediction problem. The paper demonstrates the usefulness of the proposed approach via extensive evaluation and ablation analysis over several benchmark datasets.
  arXiv  Detail & Related papers  (2023-02-13T17:55:58Z)
• SfM-TTR: Using Structure from Motion for Test-Time Refinement of Single-View Depth Networks [13.249453757295086]
  We propose a novel test-time refinement (TTR) method, denoted as SfM-TTR, to boost the performance of single-view depth networks at test time. Specifically, and differently from the state of the art, we use sparse SfM point clouds as test-time self-supervisory signal. Our results show how the addition of SfM-TTR to several state-of-the-art self-supervised and supervised networks improves significantly their performance.
  arXiv  Detail & Related papers  (2022-11-24T12:02:13Z)
• Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry [25.003116148843525]
  We propose MaGNet, a framework for fusing single-view depth probability with multi-view geometry. MaGNet achieves state-of-the-art performance on ScanNet, 7-Scenes and KITTI.
  arXiv  Detail & Related papers  (2021-12-15T14:56:53Z)
• CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth [83.77839773394106]
  We present a lightweight, tightly-coupled deep depth network and visual-inertial odometry system. We provide the network with previously marginalized sparse features from VIO to increase the accuracy of initial depth prediction. We show that it can run in real-time with single-thread execution while utilizing GPU acceleration only for the network and code Jacobian.
  arXiv  Detail & Related papers  (2020-12-18T09:42:54Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.