Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
- URL: http://arxiv.org/abs/2210.10108v1
- Date: Tue, 18 Oct 2022 19:09:58 GMT
- Title: Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
- Authors: Yunzhi Lin, Thomas M\"uller, Jonathan Tremblay, Bowen Wen, Stephen
Tyree, Alex Evans, Patricio A. Vela, Stan Birchfield
- Abstract summary: We present a parallelized optimization method based on fast Neural Radiance Fields (NeRF) for estimating 6-DoF target poses.
We can predict the translation and rotation of the camera by minimizing the residual between pixels rendered from a fast NeRF model and pixels in the observed image.
Experiments demonstrate that our method can achieve improved generalization and robustness on both synthetic and real-world benchmarks.
- Score: 26.987638406423123
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a parallelized optimization method based on fast Neural Radiance
Fields (NeRF) for estimating 6-DoF target poses. Given a single observed RGB
image of the target, we can predict the translation and rotation of the camera
by minimizing the residual between pixels rendered from a fast NeRF model and
pixels in the observed image. We integrate a momentum-based camera extrinsic
optimization procedure into Instant Neural Graphics Primitives, a recent
exceptionally fast NeRF implementation. By introducing parallel Monte Carlo
sampling into the pose estimation task, our method overcomes local minima and
improves efficiency in a more extensive search space. We also show the
importance of adopting a more robust pixel-based loss function to reduce error.
Experiments demonstrate that our method can achieve improved generalization and
robustness on both synthetic and real-world benchmarks.
Related papers
- LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors.
We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption.
LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
arXiv Detail & Related papers (2024-07-13T16:09:45Z) - Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications [83.8743080143778]
The integration of omnidirectional cameras, offering a larger field of view compared to traditional RGB cameras, has proven to yield more accurate and robust results.
However, challenges arise in situations that lack features, have substantial noise causing significant errors, and where certain features in the images lack sufficient strength.
Here, we address these challenges by introducing a novel visual gyroscope, which combines an analytical method with a neural network approach.
arXiv Detail & Related papers (2024-04-02T13:19:06Z) - BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance
Fields [0.0]
We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization.
NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
arXiv Detail & Related papers (2023-10-05T14:27:06Z) - NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor
Multi-view Stereo [97.07453889070574]
We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors.
We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
arXiv Detail & Related papers (2021-09-02T17:54:31Z) - FasterPose: A Faster Simple Baseline for Human Pose Estimation [65.8413964785972]
We propose a design paradigm for cost-effective network with LR representation for efficient pose estimation, named FasterPose.
We study the training behavior of FasterPose, and formulate a novel regressive cross-entropy (RCE) loss function for accelerating the convergence.
Compared with the previously dominant network of pose estimation, our method reduces 58% of the FLOPs and simultaneously gains 1.3% improvement of accuracy.
arXiv Detail & Related papers (2021-07-07T13:39:08Z) - ROSEFusion: Random Optimization for Online Dense Reconstruction under
Fast Camera Motion [15.873973449155313]
Reconstruction based on RGB-D sequences has thus far been restrained to relatively slow camera motions (1m/s)
Fast motion brings two challenges to depth fusion: 1) the high nonlinearity of camera pose optimization due to large inter-frame rotations and 2) the lack of reliably trackable features due to motion blur.
We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization.
Thanks to the efficient template-based particle set evolution and the effective fitness function, our method attains good quality pose tracking under fast camera motion (up to 4m/s) in a
arXiv Detail & Related papers (2021-05-12T11:37:34Z) - Robust 360-8PA: Redesigning The Normalized 8-point Algorithm for 360-FoV
Images [53.11097060367591]
We present a novel strategy for estimating an essential matrix from 360-FoV images in spherical projection.
We show that our normalization can increase the camera pose accuracy by about 20% without significantly overhead the time.
arXiv Detail & Related papers (2021-04-22T07:23:11Z) - Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene.
We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images.
Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
arXiv Detail & Related papers (2020-12-18T08:25:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.