Edge-aware Consistent Stereo Video Depth Estimation
- URL: http://arxiv.org/abs/2305.02645v1
- Date: Thu, 4 May 2023 08:30:04 GMT
- Title: Edge-aware Consistent Stereo Video Depth Estimation
- Authors: Elena Kosheleva, Sunil Jaiswal, Faranak Shamsafar, Noshaba Cheema,
Klaus Illgner-Fehns, Philipp Slusallek
- Abstract summary: We propose a consistent method for dense video depth estimation.
Unlike the existing monocular methods, ours relates to stereo videos.
We show that our edge-aware stereo video model can accurately estimate the dense depth maps.
- Score: 3.611754783778107
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Video depth estimation is crucial in various applications, such as scene
reconstruction and augmented reality. In contrast to the naive method of
estimating depths from images, a more sophisticated approach uses temporal
information, thereby eliminating flickering and geometrical inconsistencies. We
propose a consistent method for dense video depth estimation; however, unlike
the existing monocular methods, ours relates to stereo videos. This technique
overcomes the limitations arising from the monocular input. As a benefit of
using stereo inputs, a left-right consistency loss is introduced to improve the
performance. Besides, we use SLAM-based camera pose estimation in the process.
To address the problem of depth blurriness during test-time training (TTT), we
present an edge-preserving loss function that improves the visibility of fine
details while preserving geometrical consistency. We show that our edge-aware
stereo video model can accurately estimate the dense depth maps.
Related papers
- Pixel-Aligned Multi-View Generation with Depth Guided Decoder [86.1813201212539]
We propose a novel method for pixel-level image-to-multi-view generation.
Unlike prior work, we incorporate attention layers across multi-view images in the VAE decoder of a latent video diffusion model.
Our model enables better pixel alignment across multi-view images.
arXiv Detail & Related papers (2024-08-26T04:56:41Z) - Learning Temporally Consistent Video Depth from Video Diffusion Priors [57.929828486615605]
This work addresses the challenge of video depth estimation.
We reformulate the prediction task into a conditional generation problem.
This allows us to leverage the prior knowledge embedded in existing video generation models.
arXiv Detail & Related papers (2024-06-03T16:20:24Z) - SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets [65.64958606221069]
Multi-camera systems are often used in autonomous driving to achieve a 360$circ$ perception.
These 360$circ$ camera sets often have limited or low-quality overlap regions, making multi-view stereo methods infeasible for the entire image.
We propose the Stereo Guided Depth Estimation (SGDE) method, which enhances depth estimation of the full image by explicitly utilizing multi-view stereo results on the overlap.
arXiv Detail & Related papers (2024-02-19T02:41:37Z) - Temporally Consistent Online Depth Estimation Using Point-Based Fusion [6.5514240555359455]
We aim to estimate temporally consistent depth maps of video streams in an online setting.
This is a difficult problem as future frames are not available and the method must choose between enforcing consistency and correcting errors from previous estimations.
We propose to address these challenges by using a global point cloud that is dynamically updated each frame, along with a learned fusion approach in image space.
arXiv Detail & Related papers (2023-04-15T00:04:18Z) - DEVO: Depth-Event Camera Visual Odometry in Challenging Conditions [30.892930944644853]
We present a novel real-time visual odometry framework for a stereo setup of a depth and high-resolution event camera.
Our framework balances accuracy and robustness against computational efficiency towards strong performance in challenging scenarios.
arXiv Detail & Related papers (2022-02-05T13:46:47Z) - Depth Refinement for Improved Stereo Reconstruction [13.941756438712382]
Current techniques for depth estimation from stereoscopic images still suffer from a built-in drawback.
A simple analysis reveals that the depth error is quadratically proportional to the object's distance.
We propose a simple but effective method that uses a refinement network for depth estimation.
arXiv Detail & Related papers (2021-12-15T12:21:08Z) - Robust Consistent Video Depth Estimation [65.53308117778361]
We present an algorithm for estimating consistent dense depth maps and camera poses from a monocular video.
Our algorithm combines two complementary techniques: (1) flexible deformation-splines for low-frequency large-scale alignment and (2) geometry-aware depth filtering for high-frequency alignment of fine depth details.
In contrast to prior approaches, our method does not require camera poses as input and achieves robust reconstruction for challenging hand-held cell phone captures containing a significant amount of noise, shake, motion blur, and rolling shutter deformations.
arXiv Detail & Related papers (2020-12-10T18:59:48Z) - Self-Attention Dense Depth Estimation Network for Unrectified Video
Sequences [6.821598757786515]
LiDAR and radar sensors are the hardware solution for real-time depth estimation.
Deep learning based self-supervised depth estimation methods have shown promising results.
We propose a self-attention based depth and ego-motion network for unrectified images.
arXiv Detail & Related papers (2020-05-28T21:53:53Z) - Consistent Video Depth Estimation [57.712779457632024]
We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video.
We leverage a conventional structure-from-motion reconstruction to establish geometric constraints on pixels in the video.
Our algorithm is able to handle challenging hand-held captured input videos with a moderate degree of dynamic motion.
arXiv Detail & Related papers (2020-04-30T17:59:26Z) - Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
We present a new learning-based method for multi-frame depth estimation from a color video.
Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
arXiv Detail & Related papers (2020-04-02T07:10:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.