GSFeatLoc: Visual Localization Using Feature Correspondence on 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2504.20379v2
• Date: Thu, 01 May 2025 02:33:42 GMT
• Title: GSFeatLoc: Visual Localization Using Feature Correspondence on 3D Gaussian Splatting
• Authors: Jongwon Lee, Timothy Bretl,
• Abstract summary: We present a method for localizing a query image with respect to a precomputed 3D scene representation.<n>Results show that our method significantly reduces both inference time and estimation error.<n>Results also show that our method tolerates large errors in the initial pose estimate of up to 55deg in rotation and 1.1 units in translation.
• Score: 3.68055792519924
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present a method for localizing a query image with respect to a precomputed 3D Gaussian Splatting (3DGS) scene representation. First, the method uses 3DGS to render a synthetic RGBD image at some initial pose estimate. Second, it establishes 2D-2D correspondences between the query image and this synthetic image. Third, it uses the depth map to lift the 2D-2D correspondences to 2D-3D correspondences and solves a perspective-n-point (PnP) problem to produce a final pose estimate. Results from evaluation across three existing datasets with 38 scenes and over 2,700 test images show that our method significantly reduces both inference time (by over two orders of magnitude, from more than 10 seconds to as fast as 0.1 seconds) and estimation error compared to baseline methods that use photometric loss minimization. Results also show that our method tolerates large errors in the initial pose estimate of up to 55{\deg} in rotation and 1.1 units in translation (normalized by scene scale), achieving final pose errors of less than 5{\deg} in rotation and 0.05 units in translation on 90% of images from the Synthetic NeRF and Mip-NeRF360 datasets and on 42% of images from the more challenging Tanks and Temples dataset.

Related papers

• SGLoc: Semantic Localization System for Camera Pose Estimation from 3D Gaussian Splatting Representation [9.77843053500054]
  We propose SGLoc, a novel localization system that directly regresses camera poses from 3D Gaussian Splatting (3DGS) representation by leveraging semantic information.<n>Our method utilizes the semantic relationship between 2D image and 3D scene representation to estimate the 6DoF pose without prior pose information.
  arXiv  Detail & Related papers  (2025-07-16T08:39:08Z)
• Dense 3D Displacement Estimation for Landslide Monitoring via Fusion of TLS Point Clouds and Embedded RGB Images [7.144866519844918]
  Landslide monitoring is essential for understanding geohazards and mitigating associated risks.<n>Existing point cloud-based methods typically rely on either geometric or radiometric information.<n>We propose a hierarchical partition-based coarse-to-fine approach that fuses 3D point clouds and co-registered RGB images.
  arXiv  Detail & Related papers  (2025-06-19T12:28:09Z)
• No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images [100.80376573969045]
  NoPoSplat is a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from multi-view images. Our model achieves real-time 3D Gaussian reconstruction during inference. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-31T17:58:22Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices.<n>Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization [1.4466437171584356]
  We propose a two-stage procedure that integrates dense and robust keypoint descriptors from the lightweight XFeat feature extractor into 3DGS.<n>In the second stage, the initial pose estimate is refined by minimizing the rendering-based photometric warp loss.<n> Benchmarking on widely used indoor and outdoor datasets demonstrates improvements over recent neural rendering-based localization methods.
  arXiv  Detail & Related papers  (2024-09-24T23:18:32Z)
• Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion [20.464224937528222]
  Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS suffers from a well-known'missing cone' problem, which results in poor reconstruction along the depth axis. We propose fusion algorithms that simultaneously utilize RGB camera data and sonar data.
  arXiv  Detail & Related papers  (2024-04-06T17:23:43Z)
• InstantSplat: Sparse-view Gaussian Splatting in Seconds [91.77050739918037]
  We introduce InstantSplat, a novel approach for addressing sparse-view 3D scene reconstruction at lightning-fast speed.<n>InstantSplat employs a self-supervised framework that optimize 3D scene representation and camera poses.<n>It achieves an acceleration of over 30x in reconstruction and improves visual quality (SSIM) from 0.3755 to 0.7624 compared to traditional SfM with 3D-GS.
  arXiv  Detail & Related papers  (2024-03-29T17:29:58Z)
• Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images [47.14713579719103]
  We introduce a dense depth map as a geometry guide to mitigate overfitting. The adjusted depth aids in the color-based optimization of 3D Gaussian splatting. We verify the proposed method on the NeRF-LLFF dataset with varying numbers of few images.
  arXiv  Detail & Related papers  (2023-11-22T13:53:04Z)
• EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual Localization [44.05930316729542]
  We propose EP2P-Loc, a novel large-scale visual localization method for 3D point clouds. To increase the number of inliers, we propose a simple algorithm to remove invisible 3D points in the image. For the first time in this task, we employ a differentiable for end-to-end training.
  arXiv  Detail & Related papers  (2023-09-14T07:06:36Z)
• CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Network [66.24726878647543]
  Estimating the 6-DoF pose of a rigid object from a single RGB image is a crucial yet challenging task. Recent studies have shown the great potential of dense correspondence-based solutions. We propose a novel pose estimation algorithm named CheckerPose, which improves on three main aspects.
  arXiv  Detail & Related papers  (2023-03-29T17:30:53Z)
• LFM-3D: Learnable Feature Matching Across Wide Baselines Using 3D Signals [9.201550006194994]
  Learnable matchers often underperform when there exists only small regions of co-visibility between image pairs. We propose LFM-3D, a Learnable Feature Matching framework that uses models based on graph neural networks. We show that the resulting improved correspondences lead to much higher relative posing accuracy for in-the-wild image pairs.
  arXiv  Detail & Related papers  (2023-03-22T17:46:27Z)
• SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling [75.957103837167]
  Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch.
  arXiv  Detail & Related papers  (2022-08-14T16:37:51Z)
• Uncertainty-Aware Camera Pose Estimation from Points and Lines [101.03675842534415]
  Perspective-n-Point-and-Line (Pn$PL) aims at fast, accurate and robust camera localizations with respect to a 3D model from 2D-3D feature coordinates.
  arXiv  Detail & Related papers  (2021-07-08T15:19:36Z)
• Synthetic Training for Monocular Human Mesh Recovery [100.38109761268639]
  This paper aims to estimate 3D mesh of multiple body parts with large-scale differences from a single RGB image. The main challenge is lacking training data that have complete 3D annotations of all body parts in 2D images. We propose a depth-to-scale (D2S) projection to incorporate the depth difference into the projection function to derive per-joint scale variants.
  arXiv  Detail & Related papers  (2020-10-27T03:31:35Z)
• Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild [96.09941542587865]
  We present a novel 3D pose refinement approach based on differentiable rendering for objects of arbitrary categories in the wild. In this way, we precisely align 3D models to objects in RGB images which results in significantly improved 3D pose estimates. We evaluate our approach on the challenging Pix3D dataset and achieve up to 55% relative improvement compared to state-of-the-art refinement methods in multiple metrics.
  arXiv  Detail & Related papers  (2020-07-17T12:34:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.