NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features

• URL: http://arxiv.org/abs/2503.06117v1
• Date: Sat, 08 Mar 2025 08:04:27 GMT
• Title: NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features
• Authors: Hongjia Zhai, Boming Zhao, Hai Li, Xiaokun Pan, Yijia He, Zhaopeng Cui, Hujun Bao, Guofeng Zhang,
• Abstract summary: We propose an efficient and novel visual localization approach based on the neural implicit map with complementary features.<n>Specifically, to enforce geometric constraints and reduce storage requirements, we implicitly learn a 3D keypoint descriptor field.<n>To further address the semantic ambiguity of descriptors, we introduce additional semantic contextual feature fields.
• Score: 50.212836834889146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, neural radiance fields (NeRF) have gained significant attention in the field of visual localization. However, existing NeRF-based approaches either lack geometric constraints or require extensive storage for feature matching, limiting their practical applications. To address these challenges, we propose an efficient and novel visual localization approach based on the neural implicit map with complementary features. Specifically, to enforce geometric constraints and reduce storage requirements, we implicitly learn a 3D keypoint descriptor field, avoiding the need to explicitly store point-wise features. To further address the semantic ambiguity of descriptors, we introduce additional semantic contextual feature fields, which enhance the quality and reliability of 2D-3D correspondences. Besides, we propose descriptor similarity distribution alignment to minimize the domain gap between 2D and 3D feature spaces during matching. Finally, we construct the matching graph using both complementary descriptors and contextual features to establish accurate 2D-3D correspondences for 6-DoF pose estimation. Compared with the recent NeRF-based approaches, our method achieves a 3$\times$ faster training speed and a 45$\times$ reduction in model storage. Extensive experiments on two widely used datasets demonstrate that our approach outperforms or is highly competitive with other state-of-the-art NeRF-based visual localization methods. Project page: \href{https://zju3dv.github.io/neuraloc}{https://zju3dv.github.io/neuraloc}

Related papers

• Cues3D: Unleashing the Power of Sole NeRF for Consistent and Unique Instances in Open-Vocabulary 3D Panoptic Segmentation [48.231573110948]
  Open-vocabulary 3D panoptic segmentation has recently emerged as a significant trend. We present Cues3D, a compact approach that relies solely on Neural Radiance Field (NeRF) instead of pre-associations. Our experiments are conducted on ScanNet v2, ScanNet200, ScanNet++, and Replica datasets for 3D instance, panoptic, and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2025-05-01T08:12:03Z)
• SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality [50.179377002092416]
  We propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches.
  arXiv  Detail & Related papers  (2024-09-21T08:46:16Z)
• Leveraging Neural Radiance Field in Descriptor Synthesis for Keypoints Scene Coordinate Regression [1.2974519529978974]
  This paper introduces a pipeline for keypoint descriptor synthesis using Neural Radiance Field (NeRF) generating novel poses and feeding them into a trained NeRF model to create new views, our approach enhances the KSCR's capabilities in data-scarce environments. The proposed system could significantly improve localization accuracy by up to 50% and cost only a fraction of time for data synthesis.
  arXiv  Detail & Related papers  (2024-03-15T13:40:37Z)
• Improved Scene Landmark Detection for Camera Localization [11.56648898250606]
  Method based on scene landmark detection (SLD) was recently proposed to address these limitations. It involves training a convolutional neural network (CNN) to detect a few predetermined, salient, scene-specific 3D points or landmarks. We show that the accuracy gap was due to insufficient model capacity and noisy labels during training.
  arXiv  Detail & Related papers  (2024-01-31T18:59:12Z)
• Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields [54.482261428543985]
  Methods that use Neural Radiance fields are versatile for traditional tasks such as novel view synthesis. 3D Gaussian splatting has shown state-of-the-art performance on real-time radiance field rendering. We propose architectural and training changes to efficiently avert this problem.
  arXiv  Detail & Related papers  (2023-12-06T00:46:30Z)
• ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction [62.599588577671796]
  We propose an online 3D semantic segmentation method that incrementally reconstructs a 3D semantic map from a stream of RGB-D frames. Unlike offline methods, ours is directly applicable to scenarios with real-time constraints, such as robotics or mixed reality.
  arXiv  Detail & Related papers  (2023-11-29T20:30:18Z)
• DGC-GNN: Leveraging Geometry and Color Cues for Visual Descriptor-Free 2D-3D Matching [39.461400537109895]
  Matching 2D keypoints in an image to a sparse 3D point cloud of the scene without requiring visual descriptors has garnered increased interest. We introduce DGC-GNN, a novel algorithm that exploits geometric and color cues to represent keypoints, thereby improving matching accuracy. We evaluate DGC-GNN on both indoor and outdoor datasets, demonstrating that it not only doubles the accuracy of the state-of-the-art visual descriptor-free algorithm but also substantially narrows the performance gap between descriptor-based and descriptor-free methods.
  arXiv  Detail & Related papers  (2023-06-21T20:21:15Z)
• NeRF-Loc: Visual Localization with Conditional Neural Radiance Field [25.319374695362267]
  We propose a novel visual re-localization method based on direct matching between implicit 3D descriptors and the 2D image with transformer. Experiments show that our method achieves higher localization accuracy than other learning-based approaches on multiple benchmarks.
  arXiv  Detail & Related papers  (2023-04-17T03:53:02Z)
• Learning Smooth Neural Functions via Lipschitz Regularization [92.42667575719048]
  We introduce a novel regularization designed to encourage smooth latent spaces in neural fields. Compared with prior Lipschitz regularized networks, ours is computationally fast and can be implemented in four lines of code.
  arXiv  Detail & Related papers  (2022-02-16T21:24:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.