SG2Loc: Sequential Visual Localization on 3D Scene Graphs

• URL: http://arxiv.org/abs/2606.11880v1
• Date: Wed, 10 Jun 2026 10:03:21 GMT
• Title: SG2Loc: Sequential Visual Localization on 3D Scene Graphs
• Abstract summary: This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs.<n>By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world datasets.
• Score: 108.42363585468458
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual localization in complex indoor environments remains a critical challenge for robotics and AR applications. Sequential localization, where pose estimates are refined over time, is important for autonomous agents. However, traditional methods often require storing extensive image databases or point clouds, leading to significant overhead. This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs. Our method represents the environment with a compact scene graph, where nodes represent objects (with coarse meshes) and edges encode spatial relationships. For each image in the localization phase, we extract per-patch semantic features, predicting object identities. Localization is performed within a particle filter framework. Each particle, representing a camera pose, projects the coarse object meshes from the scene graph into the image, assigning object identities to patches based on visibility. The similarity of the per-patch features, in the input image, and object features from the scene graph determines the weight of a particle. Subsequent images are incorporated sequentially, refining the pose estimate. By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world datasets. The code will be available at https://github.com/DmblnNicole/sg2loc.

Related papers

• Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs [10.390999817790425]
  Open-vocabulary 3D scene graph methods typically operate in two stages: first reconstruct, then enrich with vision-language models.<n>We propose an asynchronous architecture in which lightweight online mapping runs concurrently with heavyweight semantic refinement.<n>Our method matches or outperforms existing open-vocabulary 3D scene graph methods on semantic segmentation.
  arXiv  Detail & Related papers  (2026-06-30T10:49:21Z)
• SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching [2.4737219202679808]
  We propose SceneGrapher, a framework that reformulates 3D grounding as structured graph matching over a reconstructed 3D scene graph.<n>We demonstrate our framework through real-world deployment on a mobile robot, demonstrating robust spatial reasoning in long-horizon physical environments.
  arXiv  Detail & Related papers  (2026-05-20T22:30:51Z)
• Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization [30.56844309008454]
  We propose a scene representation for visual localization that combines an explicit geometry model (3DGS) with an implicit feature field.<n>We use a 3D structure-informed clustering procedure to regularize the representation learning and seamlessly convert the features to segmentations.<n>The resulting privacy- and non-privacy-preserving localization pipelines, evaluated on multiple real-world datasets, show state-of-the-art performances.
  arXiv  Detail & Related papers  (2025-07-31T13:58:15Z)
• Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement [32.335953514942474]
  This paper proposes to jointly learn the scene representation along with a 3D dense feature field and a 2D feature extractor. We learn the underlying geometry of the scene with an implicit field through volumetric rendering and design our feature field to leverage intermediate geometric information encoded in the implicit field. Visual localization is then achieved by aligning the image-based features and the rendered volumetric features.
  arXiv  Detail & Related papers  (2024-06-12T17:51:53Z)
• SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs [81.2396059480232]
  SceneGraphLoc learns a fixed-sized embedding for each node (i.e., representing an object instance) in the scene graph. When images are leveraged, SceneGraphLoc achieves performance close to that of state-of-the-art techniques depending on large image databases.
  arXiv  Detail & Related papers  (2024-03-30T20:25:16Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)
• Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training [112.94542676251133]
  We propose to learn scene graph embeddings by directly optimizing their alignment with images. Specifically, we pre-train an encoder to extract both global and local information from scene graphs. The resulting method, called SGDiff, allows for the semantic manipulation of generated images by modifying scene graph nodes and connections.
  arXiv  Detail & Related papers  (2022-11-21T01:11:19Z)
• MeshLoc: Mesh-Based Visual Localization [54.731309449883284]
  We explore a more flexible alternative based on dense 3D meshes that does not require features matching between database images to build the scene representation. Surprisingly competitive results can be obtained when extracting features on renderings of these meshes, without any neural rendering stage. Our results show that dense 3D model-based representations are a promising alternative to existing representations and point to interesting and challenging directions for future research.
  arXiv  Detail & Related papers  (2022-07-21T21:21:10Z)
• Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions [94.17683799712397]
  We focus on scene graphs, a data structure that organizes the entities of a scene in a graph. We propose a learned method that regresses a scene graph from the point cloud of a scene. We show the application of our method in a domain-agnostic retrieval task, where graphs serve as an intermediate representation for 3D-3D and 2D-3D matching.
  arXiv  Detail & Related papers  (2020-04-08T12:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.