Matching Query Image Against Selected NeRF Feature for Efficient and Scalable Localization

• URL: http://arxiv.org/abs/2406.11766v1
• Date: Mon, 17 Jun 2024 17:29:21 GMT
• Title: Matching Query Image Against Selected NeRF Feature for Efficient and Scalable Localization
• Authors: Huaiji Zhou, Bing Wang, Changhao Chen,
• Abstract summary: This work proposes MatLoc-NeRF, a novel matching-based localization framework using selected NeRF features. It addresses efficiency by employing a learnable feature selection mechanism that identifies informative NeRF features for matching with query images. Evaluations on public large-scale datasets demonstrate that MatLoc-NeRF achieves superior efficiency and accuracy compared to existing NeRF-based localization methods.
• Score: 6.0557193420601525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural implicit representations such as NeRF have revolutionized 3D scene representation with photo-realistic quality. However, existing methods for visual localization within NeRF representations suffer from inefficiency and scalability issues, particularly in large-scale environments. This work proposes MatLoc-NeRF, a novel matching-based localization framework using selected NeRF features. It addresses efficiency by employing a learnable feature selection mechanism that identifies informative NeRF features for matching with query images. This eliminates the need for all NeRF features or additional descriptors, leading to faster and more accurate pose estimation. To tackle large-scale scenes, MatLoc-NeRF utilizes a pose-aware scene partitioning strategy. It ensures that only the most relevant NeRF sub-block generates key features for a specific pose. Additionally, scene segmentation and a place predictor provide fast coarse initial pose estimation. Evaluations on public large-scale datasets demonstrate that MatLoc-NeRF achieves superior efficiency and accuracy compared to existing NeRF-based localization methods.

Related papers

• Rate-aware Compression for NeRF-based Volumetric Video [21.372568857027748]
  radiance fields (NeRF) have advanced the development of 3D volumetric video technology. Existing solutions compress NeRF representations after the training stage, leading to a separation between representation training and compression. In this paper, we try to directly learn a compact NeRF representation for volumetric video in the training stage based on the proposed rate-aware compression framework.
  arXiv  Detail & Related papers  (2024-11-08T04:29:14Z)
• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• IFFNeRF: Initialisation Free and Fast 6DoF pose estimation from a single image and a NeRF model [17.202161109957608]
  We introduce IFFNeRF to estimate the six degrees-of-freedom (6DoF) camera pose of a given image. IFFNeRF is specifically designed to operate in real-time and eliminates the need for an initial pose guess.
  arXiv  Detail & Related papers  (2024-03-19T12:36:51Z)
• 3D Visibility-aware Generalizable Neural Radiance Fields for Interacting Hands [51.305421495638434]
  Neural radiance fields (NeRFs) are promising 3D representations for scenes, objects, and humans. This paper proposes a generalizable visibility-aware NeRF framework for interacting hands. Experiments on the Interhand2.6M dataset demonstrate that our proposed VA-NeRF outperforms conventional NeRFs significantly.
  arXiv  Detail & Related papers  (2024-01-02T00:42:06Z)
• Informative Rays Selection for Few-Shot Neural Radiance Fields [0.3599866690398789]
  KeyNeRF is a simple yet effective method for training NeRF in few-shot scenarios by focusing on key informative rays. Our approach performs favorably against state-of-the-art methods, while requiring minimal changes to existing NeRFs.
  arXiv  Detail & Related papers  (2023-12-29T11:08:19Z)
• PNeRFLoc: Visual Localization with Point-based Neural Radiance Fields [54.8553158441296]
  We propose a novel visual localization framework, ie, PNeRFLoc, based on a unified point-based representation. On the one hand, PNeRFLoc supports the initial pose estimation by matching 2D and 3D feature points. On the other hand, it also enables pose refinement with novel view synthesis using rendering-based optimization.
  arXiv  Detail & Related papers  (2023-12-17T08:30:00Z)
• LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs [56.050550636941836]
  A critical obstacle preventing NeRF models from being deployed broadly in the wild is their reliance on accurate camera poses. We propose a novel approach, LU-NeRF, that jointly estimates camera poses and neural fields with relaxed assumptions on pose configuration. We show our LU-NeRF pipeline outperforms prior attempts at unposed NeRF without making restrictive assumptions on the pose prior.
  arXiv  Detail & Related papers  (2023-06-08T17:56:22Z)
• NeRFuser: Large-Scale Scene Representation by NeRF Fusion [35.749208740102546]
  A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency. We propose NeRFuser, a novel architecture for NeRF registration and blending that assumes only access to pre-generated NeRFs.
  arXiv  Detail & Related papers  (2023-05-22T17:59:05Z)
• SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction [147.9379707578091]
  NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks. Despite NeRFmm producing precise scene synthesis and pose estimations, it still struggles to outperform the full-annotated baseline on challenging scenes. We propose Sinusoidal Neural Radiance Fields (SiNeRF) that leverage sinusoidal activations for radiance mapping and a novel Mixed Region Sampling (MRS) for selecting ray batch efficiently.
  arXiv  Detail & Related papers  (2022-10-10T10:47:51Z)
• Recursive-NeRF: An Efficient and Dynamically Growing NeRF [34.768382663711705]
  Recursive-NeRF is an efficient rendering and training approach for the Neural Radiance Field (NeRF) method. Recursive-NeRF learns uncertainties for query coordinates, representing the quality of the predicted color and volumetric intensity at each level. Our evaluation on three public datasets shows that Recursive-NeRF is more efficient than NeRF while providing state-of-the-art quality.
  arXiv  Detail & Related papers  (2021-05-19T12:51:54Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.