Movement-induced Priors for Deep Stereo

• URL: http://arxiv.org/abs/2010.09105v1
• Date: Sun, 18 Oct 2020 21:02:25 GMT
• Title: Movement-induced Priors for Deep Stereo
• Abstract summary: We propose a method for fusing stereo disparity estimation with movement-induced prior information. We present a hierarchy of three Gaussian process kernels depending on the availability of motion information. We show how our method can be combined with two state-of-the-art deep stereo methods.
• Score: 27.098315663348814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a method for fusing stereo disparity estimation with movement-induced prior information. Instead of independent inference frame-by-frame, we formulate the problem as a non-parametric learning task in terms of a temporal Gaussian process prior with a movement-driven kernel for inter-frame reasoning. We present a hierarchy of three Gaussian process kernels depending on the availability of motion information, where our main focus is on a new gyroscope-driven kernel for handheld devices with low-quality MEMS sensors, thus also relaxing the requirement of having full 6D camera poses available. We show how our method can be combined with two state-of-the-art deep stereo methods. The method either work in a plug-and-play fashion with pre-trained deep stereo networks, or further improved by jointly training the kernels together with encoder-decoder architectures, leading to consistent improvement.

Related papers

• Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos [9.965683509488203]
  This paper presents a unified framework that allows high-quality dynamic Gaussian Splatting from both defocused and motion-red monocular videos.<n>We use a blur prediction network that exploits blur-related scene and camera information and is subject to a blur-aware sparsity constraint.<n>Our method outperforms the state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-red monocular videos.
  arXiv  Detail & Related papers  (2025-10-12T16:38:54Z)
• G-CUT3R: Guided 3D Reconstruction with Camera and Depth Prior Integration [57.67450930037339]
  We introduce G-CUT3R, a novel feed-forward approach for guided 3D scene reconstruction.<n>Unlike existing feed-forward methods that rely solely on input images, our method leverages auxiliary data, such as depth, camera calibrations, or camera positions.
  arXiv  Detail & Related papers  (2025-08-15T10:25:58Z)
• Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model [62.37493746544967]
  Camera-based setups offer a cost-effective option by using stereo depth estimation to generate dense, high-resolution depth maps. Existing omnidirectional stereo matching approaches achieve only limited depth accuracy across diverse environments. We present DFI-OmniStereo, a novel omnidirectional stereo matching method that leverages a large-scale pre-trained foundation model for relative monocular depth estimation.
  arXiv  Detail & Related papers  (2025-03-30T16:24:22Z)
• A Two-Stage Progressive Pre-training using Multi-Modal Contrastive Masked Autoencoders [5.069884983892437]
  We propose a new progressive pre-training method for image understanding tasks which leverages RGB-D datasets. In the first stage, we pre-train the model using contrastive learning to learn cross-modal representations. In the second stage, we further pre-train the model using masked autoencoding and denoising/noise prediction. Our approach is scalable, robust and suitable for pre-training RGB-D datasets.
  arXiv  Detail & Related papers  (2024-08-05T05:33:59Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformers [57.46911575980854]
  We introduce SkelFormer, a novel markerless motion capture pipeline for multi-view human pose and shape estimation. Our method first uses off-the-shelf 2D keypoint estimators, pre-trained on large-scale in-the-wild data, to obtain 3D joint positions. Next, we design a regression-based inverse-kinematic skeletal transformer that maps the joint positions to pose and shape representations from heavily noisy observations.
  arXiv  Detail & Related papers  (2024-04-19T04:51:18Z)
• Trajectory Approximation of Video Based on Phase Correlation for Forward Facing Camera [0.0]
  We introduce an innovative approach for extracting trajectories from a camera sensor in GPS-denied environments, leveraging visual odometry. The system takes video footage captured by a forward-facing camera mounted on a vehicle as input, with the output being a chain code representing the camera's trajectory. Notably, our system outperforms traditional methods reliant on spatial features, exhibiting greater speed and robustness in noisy environments.
  arXiv  Detail & Related papers  (2023-12-20T00:44:04Z)
• Neural Disparity Refinement for Arbitrary Resolution Stereo [67.55946402652778]
  We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution.
  arXiv  Detail & Related papers  (2021-10-28T18:00:00Z)
• Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection [86.25022248968908]
  We learn context- and depth-aware feature representation to solve the problem of monocular 3D object detection. We show state-of-the-art results among the monocular-based approaches on the KITTI benchmark dataset.
  arXiv  Detail & Related papers  (2021-03-30T16:20:24Z)
• Robust Consistent Video Depth Estimation [65.53308117778361]
  We present an algorithm for estimating consistent dense depth maps and camera poses from a monocular video. Our algorithm combines two complementary techniques: (1) flexible deformation-splines for low-frequency large-scale alignment and (2) geometry-aware depth filtering for high-frequency alignment of fine depth details. In contrast to prior approaches, our method does not require camera poses as input and achieves robust reconstruction for challenging hand-held cell phone captures containing a significant amount of noise, shake, motion blur, and rolling shutter deformations.
  arXiv  Detail & Related papers  (2020-12-10T18:59:48Z)
• Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video [10.181846237133167]
  We propose an approach to reconstruct dense three-dimensional (3D) model of tissue surface from stereo optical videos in real-time. The basic idea is to first extract 3D information from video frames by using stereo matching, and then to mosaic the reconstructed 3D models. Experimental results on ex- and in vivo data showed that the reconstructed 3D models have high resolution texture with an accuracy error of less than 2 mm.
  arXiv  Detail & Related papers  (2020-07-16T19:14:05Z)
• MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency [72.82534577726334]
  We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. Our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
  arXiv  Detail & Related papers  (2020-06-22T08:50:09Z)
• Du$^2$Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels [16.797169907541164]
  We present a novel approach based on neural networks for depth estimation that combines stereo from dual cameras with stereo from a dual-pixel sensor. Our network uses a novel architecture to fuse these two sources of information and can overcome the limitations of pure binocular stereo matching.
  arXiv  Detail & Related papers  (2020-03-31T15:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.