Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation

• URL: http://arxiv.org/abs/2210.10108v1
• Date: Tue, 18 Oct 2022 19:09:58 GMT
• Title: Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
• Authors: Yunzhi Lin, Thomas M\"uller, Jonathan Tremblay, Bowen Wen, Stephen Tyree, Alex Evans, Patricio A. Vela, Stan Birchfield
• Abstract summary: We present a parallelized optimization method based on fast Neural Radiance Fields (NeRF) for estimating 6-DoF target poses. We can predict the translation and rotation of the camera by minimizing the residual between pixels rendered from a fast NeRF model and pixels in the observed image. Experiments demonstrate that our method can achieve improved generalization and robustness on both synthetic and real-world benchmarks.
• Score: 26.987638406423123
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a parallelized optimization method based on fast Neural Radiance Fields (NeRF) for estimating 6-DoF target poses. Given a single observed RGB image of the target, we can predict the translation and rotation of the camera by minimizing the residual between pixels rendered from a fast NeRF model and pixels in the observed image. We integrate a momentum-based camera extrinsic optimization procedure into Instant Neural Graphics Primitives, a recent exceptionally fast NeRF implementation. By introducing parallel Monte Carlo sampling into the pose estimation task, our method overcomes local minima and improves efficiency in a more extensive search space. We also show the importance of adopting a more robust pixel-based loss function to reduce error. Experiments demonstrate that our method can achieve improved generalization and robustness on both synthetic and real-world benchmarks.

Related papers

• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields [0.0]
  We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
  arXiv  Detail & Related papers  (2023-10-05T14:27:06Z)
• ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution [84.73658185158222]
  We propose a diffusion model-based super-resolution method called ACDMSR. Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
  arXiv  Detail & Related papers  (2023-07-03T06:49:04Z)
• NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo [97.07453889070574]
  We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors. We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
  arXiv  Detail & Related papers  (2021-09-02T17:54:31Z)
• FasterPose: A Faster Simple Baseline for Human Pose Estimation [65.8413964785972]
  We propose a design paradigm for cost-effective network with LR representation for efficient pose estimation, named FasterPose. We study the training behavior of FasterPose, and formulate a novel regressive cross-entropy (RCE) loss function for accelerating the convergence. Compared with the previously dominant network of pose estimation, our method reduces 58% of the FLOPs and simultaneously gains 1.3% improvement of accuracy.
  arXiv  Detail & Related papers  (2021-07-07T13:39:08Z)
• ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion [15.873973449155313]
  Reconstruction based on RGB-D sequences has thus far been restrained to relatively slow camera motions (1m/s) Fast motion brings two challenges to depth fusion: 1) the high nonlinearity of camera pose optimization due to large inter-frame rotations and 2) the lack of reliably trackable features due to motion blur. We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization. Thanks to the efficient template-based particle set evolution and the effective fitness function, our method attains good quality pose tracking under fast camera motion (up to 4m/s) in a
  arXiv  Detail & Related papers  (2021-05-12T11:37:34Z)
• Robust 360-8PA: Redesigning The Normalized 8-point Algorithm for 360-FoV Images [53.11097060367591]
  We present a novel strategy for estimating an essential matrix from 360-FoV images in spherical projection. We show that our normalization can increase the camera pose accuracy by about 20% without significantly overhead the time.
  arXiv  Detail & Related papers  (2021-04-22T07:23:11Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.