VRS-NeRF: Visual Relocalization with Sparse Neural Radiance Field

• URL: http://arxiv.org/abs/2404.09271v1
• Date: Sun, 14 Apr 2024 14:26:33 GMT
• Title: VRS-NeRF: Visual Relocalization with Sparse Neural Radiance Field
• Authors: Fei Xue, Ignas Budvytis, Daniel Olmeda Reino, Roberto Cipolla,
• Abstract summary: We propose an efficient and accurate framework, called VRS-NeRF, for visual relocalization with sparse neural radiance field. In this paper, we introduce an explicit geometric map (EGM) for 3D map representation and an implicit learning map (ILM) for sparse patches rendering. Our method gives much better accuracy than APRs and SCRs, and close performance to HMs but is much more efficient.
• Score: 37.57533470759197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual relocalization is a key technique to autonomous driving, robotics, and virtual/augmented reality. After decades of explorations, absolute pose regression (APR), scene coordinate regression (SCR), and hierarchical methods (HMs) have become the most popular frameworks. However, in spite of high efficiency, APRs and SCRs have limited accuracy especially in large-scale outdoor scenes; HMs are accurate but need to store a large number of 2D descriptors for matching, resulting in poor efficiency. In this paper, we propose an efficient and accurate framework, called VRS-NeRF, for visual relocalization with sparse neural radiance field. Precisely, we introduce an explicit geometric map (EGM) for 3D map representation and an implicit learning map (ILM) for sparse patches rendering. In this localization process, EGP provides priors of spare 2D points and ILM utilizes these sparse points to render patches with sparse NeRFs for matching. This allows us to discard a large number of 2D descriptors so as to reduce the map size. Moreover, rendering patches only for useful points rather than all pixels in the whole image reduces the rendering time significantly. This framework inherits the accuracy of HMs and discards their low efficiency. Experiments on 7Scenes, CambridgeLandmarks, and Aachen datasets show that our method gives much better accuracy than APRs and SCRs, and close performance to HMs but is much more efficient.

Related papers

• NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features [50.212836834889146]
  We propose an efficient and novel visual localization approach based on the neural implicit map with complementary features. Specifically, to enforce geometric constraints and reduce storage requirements, we implicitly learn a 3D keypoint descriptor field. To further address the semantic ambiguity of descriptors, we introduce additional semantic contextual feature fields.
  arXiv  Detail & Related papers  (2025-03-08T08:04:27Z)
• GaussRender: Learning 3D Occupancy with Gaussian Rendering [86.89653628311565]
  GaussRender is a module that improves 3D occupancy learning by enforcing projective consistency. Our method penalizes 3D configurations that produce inconsistent 2D projections, thereby enforcing a more coherent 3D structure.
  arXiv  Detail & Related papers  (2025-02-07T16:07:51Z)
• SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality [50.179377002092416]
  We propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches.
  arXiv  Detail & Related papers  (2024-09-21T08:46:16Z)
• GaussReg: Fast 3D Registration with Gaussian Splatting [10.049564362260055]
  Point cloud registration is a fundamental problem for large-scale 3D scene scanning and reconstruction. We propose GaussReg, a novel coarse-to-fine framework for point cloud registration. Our method achieves state-of-the-art performance on multiple datasets.
  arXiv  Detail & Related papers  (2024-07-07T04:35:51Z)
• Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians [87.48403838439391]
  3D Splatting has emerged as a powerful representation of geometry and appearance for RGB-only dense Simultaneous SLAM. We propose the first RGB-only SLAM system with a dense 3D Gaussian map representation. Our experiments on the Replica, TUM-RGBD, and ScanNet datasets indicate the effectiveness of globally optimized 3D Gaussians.
  arXiv  Detail & Related papers  (2024-05-26T12:26:54Z)
• PRAM: Place Recognition Anywhere Model for Efficient Visual Localization [37.067966065604715]
  We propose the place recognition anywhere model (PRAM) to perform visual localization efficiently and accurately. PRAM generates 3D landmarks directly in 3D space in a self-supervised manner. It discards global descriptors, repetitive local descriptors, and redundant 3D points, increasing the memory efficiency significantly.
  arXiv  Detail & Related papers  (2024-04-11T14:28:04Z)
• Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields [54.482261428543985]
  Methods that use Neural Radiance fields are versatile for traditional tasks such as novel view synthesis. 3D Gaussian splatting has shown state-of-the-art performance on real-time radiance field rendering. We propose architectural and training changes to efficiently avert this problem.
  arXiv  Detail & Related papers  (2023-12-06T00:46:30Z)
• PyNeRF: Pyramidal Neural Radiance Fields [51.25406129834537]
  We propose a simple modification to grid-based models by training model heads at different spatial grid resolutions. At render time, we simply use coarser grids to render samples that cover larger volumes. Compared to Mip-NeRF, we reduce error rates by 20% while training over 60x faster.
  arXiv  Detail & Related papers  (2023-11-30T23:52:46Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• Fast and Lightweight Scene Regressor for Camera Relocalization [1.6708069984516967]
  Estimating the camera pose directly with respect to pre-built 3D models can be prohibitively expensive for several applications. This study proposes a simple scene regression method that requires only a multi-layer perceptron network for mapping scene coordinates. The proposed approach uses sparse descriptors to regress the scene coordinates, instead of a dense RGB image.
  arXiv  Detail & Related papers  (2022-12-04T14:41:20Z)
• PixSelect: Less but Reliable Pixels for Accurate and Efficient Localization [0.0]
  We address the problem of estimating the global 6 DoF camera pose from a single RGB image in a given environment. Our work exceeds state-ofthe-art methods on outdoor Cambridge Landmarks dataset.
  arXiv  Detail & Related papers  (2022-06-08T09:46:03Z)
• Efficient Neural Radiance Fields with Learned Depth-Guided Sampling [43.79307270743013]
  We present a hybrid scene representation which combines the best of implicit radiance fields and explicit depth maps for efficient rendering. Experiments show that the proposed approach exhibits state-of-the-art performance on the DTU, Real Forward-facing and NeRF Synthetic datasets. We also demonstrate the capability of our method to synthesize free-viewpoint videos of dynamic human performers in real-time.
  arXiv  Detail & Related papers  (2021-12-02T18:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.