LoGS: Visual Localization via Gaussian Splatting with Fewer Training Images

• URL: http://arxiv.org/abs/2410.11505v1
• Date: Tue, 15 Oct 2024 11:17:18 GMT
• Title: LoGS: Visual Localization via Gaussian Splatting with Fewer Training Images
• Authors: Yuzhou Cheng, Jianhao Jiao, Yue Wang, Dimitrios Kanoulas,
• Abstract summary: This paper presents a vision-based localization pipeline utilizing the 3D Splatting (GS) technique as scene representation. During the mapping phase, structure-from-motion (SfM) is applied first, followed by the generation of a GS map. High-precision pose is achieved through the analysis-by manner on the map.
• Score: 7.363332481155945
• License:
• Abstract: Visual localization involves estimating a query image's 6-DoF (degrees of freedom) camera pose, which is a fundamental component in various computer vision and robotic tasks. This paper presents LoGS, a vision-based localization pipeline utilizing the 3D Gaussian Splatting (GS) technique as scene representation. This novel representation allows high-quality novel view synthesis. During the mapping phase, structure-from-motion (SfM) is applied first, followed by the generation of a GS map. During localization, the initial position is obtained through image retrieval, local feature matching coupled with a PnP solver, and then a high-precision pose is achieved through the analysis-by-synthesis manner on the GS map. Experimental results on four large-scale datasets demonstrate the proposed approach's SoTA accuracy in estimating camera poses and robustness under challenging few-shot conditions.

Related papers

• NovelGS: Consistent Novel-view Denoising via Large Gaussian Reconstruction Model [57.92709692193132]
  NovelGS is a diffusion model for Gaussian Splatting given sparse-view images. We leverage the novel view denoising through a transformer-based network to generate 3D Gaussians.
  arXiv  Detail & Related papers  (2024-11-25T07:57:17Z)
• No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images [100.80376573969045]
  NoPoSplat is a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from multi-view images. Our model achieves real-time 3D Gaussian reconstruction during inference. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-31T17:58:22Z)
• GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization [1.4466437171584356]
  3D Gaussian Splatting (3DGS) allows for the compact encoding of both 3D geometry and scene appearance with its spatial features. We propose distilling dense keypoint descriptors into 3DGS to improve the model's spatial understanding. Our approach surpasses state-of-the-art Neural Render Pose (NRP) methods, including NeRFMatch and PNeRFLoc.
  arXiv  Detail & Related papers  (2024-09-24T23:18:32Z)
• Visual Localization in 3D Maps: Comparing Point Cloud, Mesh, and NeRF Representations [8.522160106746478]
  We present a global visual localization system capable of localizing a single camera image across various 3D map representations. Our system generates a database by synthesizing novel views of the scene, creating RGB and depth image pairs. NeRF synthesized images show superior performance, localizing query images at an average success rate of 72%.
  arXiv  Detail & Related papers  (2024-08-21T19:37:17Z)
• SAGS: Structure-Aware 3D Gaussian Splatting [53.6730827668389]
  We propose a structure-aware Gaussian Splatting method (SAGS) that implicitly encodes the geometry of the scene. SAGS reflects to state-of-the-art rendering performance and reduced storage requirements on benchmark novel-view synthesis datasets.
  arXiv  Detail & Related papers  (2024-04-29T23:26:30Z)
• InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds [91.77050739918037]
  Novel view synthesis (NVS) from a sparse set of images has advanced significantly in 3D computer vision. It relies on precise initial estimation of camera parameters using Structure-from-Motion (SfM) In this study, we introduce a novel and efficient framework to enhance robust NVS from sparse-view images.
  arXiv  Detail & Related papers  (2024-03-29T17:29:58Z)
• 3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization [13.868258945395326]
  This paper presents a novel system designed for 3D mapping and visual relocalization using 3D Gaussian Splatting. Our proposed method uses LiDAR and camera data to create accurate and visually plausible representations of the environment.
  arXiv  Detail & Related papers  (2024-03-17T23:06:12Z)
• GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splatting [23.724077890247834]
  GS-Pose is a framework for localizing and estimating the 6D pose of novel objects. It operates sequentially by locating the object in the input image, estimating its initial 6D pose, and refining the pose with a render-and-compare method. Off-the-shelf toolchains and commodity hardware, such as mobile phones, can be used to capture new objects to be added to the database.
  arXiv  Detail & Related papers  (2024-03-15T21:06:14Z)
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting [58.41056963451056]
  We propose a few-shot view synthesis framework based on 3D Gaussian Splatting. This framework enables real-time and photo-realistic view synthesis with as few as three training views. FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets.
  arXiv  Detail & Related papers  (2023-12-01T09:30:02Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.