Deep Probabilistic Feature-metric Tracking
- URL: http://arxiv.org/abs/2008.13504v2
- Date: Wed, 25 Nov 2020 23:47:16 GMT
- Title: Deep Probabilistic Feature-metric Tracking
- Authors: Binbin Xu, Andrew J. Davison, and Stefan Leutenegger
- Abstract summary: We propose a new framework to learn a pixel-wise deep feature map and a deep feature-metric uncertainty map.
CNN predicts a deep initial pose for faster and more reliable convergence.
Experimental results demonstrate state-of-the-art performances on the TUM RGB-D dataset and the 3D rigid object tracking dataset.
- Score: 27.137827823264942
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Dense image alignment from RGB-D images remains a critical issue for
real-world applications, especially under challenging lighting conditions and
in a wide baseline setting. In this paper, we propose a new framework to learn
a pixel-wise deep feature map and a deep feature-metric uncertainty map
predicted by a Convolutional Neural Network (CNN), which together formulate a
deep probabilistic feature-metric residual of the two-view constraint that can
be minimised using Gauss-Newton in a coarse-to-fine optimisation framework.
Furthermore, our network predicts a deep initial pose for faster and more
reliable convergence. The optimisation steps are differentiable and unrolled to
train in an end-to-end fashion. Due to its probabilistic essence, our approach
can easily couple with other residuals, where we show a combination with ICP.
Experimental results demonstrate state-of-the-art performances on the TUM RGB-D
dataset and the 3D rigid object tracking dataset. We further demonstrate our
method's robustness and convergence qualitatively.
Related papers
- $ν$-DBA: Neural Implicit Dense Bundle Adjustment Enables Image-Only Driving Scene Reconstruction [31.64067619807023]
$nu$-DBA implements geometric dense bundle adjustment (DBA) using 3D neural implicit surfaces for map parametrization.
We fine-tune the optical flow model with per-scene self-supervision to further improve the quality of the dense mapping.
arXiv Detail & Related papers (2024-04-29T05:29:26Z) - Distance Weighted Trans Network for Image Completion [52.318730994423106]
We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components.
CNNs are used to augment the local texture information of coarse priors.
DWT blocks are used to recover certain coarse textures and coherent visual structures.
arXiv Detail & Related papers (2023-10-11T12:46:11Z) - Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis [17.920305227880245]
Our paper formulates triple vision tasks in a consistent manner using approximate analysis-by-synthesis.
We show that our analysis-by-synthesis is much more robust than conventional neural networks when evaluated on real-world images.
arXiv Detail & Related papers (2023-05-31T18:45:02Z) - Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
We introduce an approach that performs continuous modeling of per-pixel depth.
Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
arXiv Detail & Related papers (2023-03-31T16:01:03Z) - A Probabilistic Deep Image Prior for Computational Tomography [0.19573380763700707]
Existing deep-learning based tomographic image reconstruction methods do not provide accurate estimates of reconstruction uncertainty.
We construct a Bayesian prior for tomographic reconstruction, which combines the classical total variation (TV) regulariser with the modern deep image prior (DIP)
For the inference, we develop an approach based on the linearised Laplace method, which is scalable to high-dimensional settings.
arXiv Detail & Related papers (2022-02-28T14:47:14Z) - PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences.
We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction.
Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
arXiv Detail & Related papers (2021-09-28T17:56:41Z) - Riggable 3D Face Reconstruction via In-Network Optimization [58.016067611038046]
This paper presents a method for riggable 3D face reconstruction from monocular images.
It jointly estimates a personalized face rig and per-image parameters including expressions, poses, and illuminations.
Experiments demonstrate that our method achieves SOTA reconstruction accuracy, reasonable robustness and generalization ability.
arXiv Detail & Related papers (2021-04-08T03:53:20Z) - Probabilistic Graph Attention Network with Conditional Kernels for
Pixel-Wise Prediction [158.88345945211185]
We present a novel approach that advances the state of the art on pixel-level prediction in a fundamental aspect, i.e. structured multi-scale features learning and fusion.
We propose a probabilistic graph attention network structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs) model for learning and fusing multi-scale representations in a principled manner.
arXiv Detail & Related papers (2021-01-08T04:14:29Z) - PaMIR: Parametric Model-Conditioned Implicit Representation for
Image-based Human Reconstruction [67.08350202974434]
We propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function.
We show that our method achieves state-of-the-art performance for image-based 3D human reconstruction in the cases of challenging poses and clothing types.
arXiv Detail & Related papers (2020-07-08T02:26:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.