BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields

• URL: http://arxiv.org/abs/2310.03563v1
• Date: Thu, 5 Oct 2023 14:27:06 GMT
• Title: BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields
• Authors: \'Agoston Istv\'an Csehi, Csaba M\'at\'e J\'ozsa
• Abstract summary: We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects. Our contributions are as follows: we extend the localization optimization objective with a depth-based loss function, we introduce a multi-image based loss function where a sequence of images with known relative poses are used without increasing the computational complexity, we omit hierarchical sampling during volumetric rendering, meaning only the coarse model is used for pose estimation, and we how that by extending the sampling interval convergence can be achieved even or higher initial pose estimate errors. With the proposed modifications the convergence speed is significantly improved, and the basin of convergence is substantially extended.

Related papers

• NODER: Image Sequence Regression Based on Neural Ordinary Differential Equations [2.711538918087856]
  We propose an optimization-based new framework called NODER, which leverages neural ordinary differential equations to capture complex underlying dynamics. Our model needs only a couple of images in a sequence for prediction, which is practical, especially for clinical situations.
  arXiv  Detail & Related papers  (2024-07-18T07:50:46Z)
• Invertible Neural Warp for NeRF [29.00183106905031]
  This paper tackles the simultaneous optimization of pose and Neural Radiance Fields (NeRF) We propose a novel over parameterized representation that models camera poses as learnable rigid warp functions. We present results on synthetic and real-world datasets, and demonstrate that our approach outperforms existing baselines in terms of pose estimation and high-fidelity reconstruction.
  arXiv  Detail & Related papers  (2024-07-17T07:14:08Z)
• Towards a Sampling Theory for Implicit Neural Representations [0.3222802562733786]
  Implicit neural representations (INRs) have emerged as a powerful tool for solving inverse problems in computer and computational imaging. We show how to recover images from a hidden-layer INR using a generalized form of weight decay regularization. We empirically assess the probability of achieving exact recovery images realized by low-width single-layer INRs, and illustrate the performance of INR on super-resolution recovery of more realistic continuous domain phantom images.
  arXiv  Detail & Related papers  (2024-05-28T17:53:47Z)
• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
  Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content. We address these issues with an approach based on ray tracing. Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
  arXiv  Detail & Related papers  (2024-05-23T17:59:57Z)
• Towards High-quality HDR Deghosting with Conditional Diffusion Models [88.83729417524823]
  High Dynamic Range (LDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. DNNs still generate ghosting artifacts when LDR images have saturation and large motion. We formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition.
  arXiv  Detail & Related papers  (2023-11-02T01:53:55Z)
• Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation [26.987638406423123]
  We present a parallelized optimization method based on fast Neural Radiance Fields (NeRF) for estimating 6-DoF target poses. We can predict the translation and rotation of the camera by minimizing the residual between pixels rendered from a fast NeRF model and pixels in the observed image. Experiments demonstrate that our method can achieve improved generalization and robustness on both synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2022-10-18T19:09:58Z)
• IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis [90.03590032170169]
  We present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method. Our experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results.
  arXiv  Detail & Related papers  (2022-10-02T22:45:11Z)
• InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering [55.70938412352287]
  We present an information-theoretic regularization technique for few-shot novel view synthesis based on neural implicit representation. The proposed approach minimizes potential reconstruction inconsistency that happens due to insufficient viewpoints. We achieve consistently improved performance compared to existing neural view synthesis methods by large margins on multiple standard benchmarks.
  arXiv  Detail & Related papers  (2021-12-31T11:56:01Z)
• NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo [97.07453889070574]
  We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors. We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
  arXiv  Detail & Related papers  (2021-09-02T17:54:31Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.