vS-Graphs: Integrating Visual SLAM and Situational Graphs through Multi-level Scene Understanding

• URL: http://arxiv.org/abs/2503.01783v1
• Date: Mon, 03 Mar 2025 18:15:11 GMT
• Title: vS-Graphs: Integrating Visual SLAM and Situational Graphs through Multi-level Scene Understanding
• Authors: Ali Tourani, Saad Ejaz, Hriday Bavle, David Morilla-Cabello, Jose Luis Sanchez-Lopez, Holger Voos,
• Abstract summary: This paper introduces visual S-Graphs (vS-Graphs), a novel real-time VSLAM framework.<n>It integrates vision-based scene understanding with map reconstruction and comprehensible graph-based representation.<n>Experiments on standard benchmarks and real-world datasets demonstrate that vS-Graphs outperforms state-of-the-art VSLAM methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current Visual Simultaneous Localization and Mapping (VSLAM) systems often struggle to create maps that are both semantically rich and easily interpretable. While incorporating semantic scene knowledge aids in building richer maps with contextual associations among mapped objects, representing them in structured formats like scene graphs has not been widely addressed, encountering complex map comprehension and limited scalability. This paper introduces visual S-Graphs (vS-Graphs), a novel real-time VSLAM framework that integrates vision-based scene understanding with map reconstruction and comprehensible graph-based representation. The framework infers structural elements (i.e., rooms and corridors) from detected building components (i.e., walls and ground surfaces) and incorporates them into optimizable 3D scene graphs. This solution enhances the reconstructed map's semantic richness, comprehensibility, and localization accuracy. Extensive experiments on standard benchmarks and real-world datasets demonstrate that vS-Graphs outperforms state-of-the-art VSLAM methods, reducing trajectory error by an average of 3.38% and up to 9.58% on real-world data. Furthermore, the proposed framework achieves environment-driven semantic entity detection accuracy comparable to precise LiDAR-based frameworks using only visual features. A web page containing more media and evaluation outcomes is available on https://snt-arg.github.io/vsgraphs-results/.

Related papers

• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding. Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• STAMICS: Splat, Track And Map with Integrated Consistency and Semantics for Dense RGB-D SLAM [8.208389210258593]
  We introduce STAMICS, a novel method that integrates semantic information with 3D Gaussian representations to enhance both localization and mapping accuracy. Experiments show that STAMICS significantly improves camera pose estimation and map quality, outperforming state-of-the-art methods while reducing reconstruction errors.
  arXiv  Detail & Related papers  (2025-03-27T12:10:51Z)
• GaussianGraph: 3D Gaussian-based Scene Graph Generation for Open-world Scene Understanding [20.578106363482018]
  We propose a novel framework that enhances 3DGS-based scene understanding by integrating semantic clustering and scene graph generation. We introduce a "Control-Follow" clustering strategy, which dynamically adapts to scene scale and feature distribution, avoiding feature compression. We enrich scene representation by integrating object attributes and spatial relations extracted from 2D foundation models.
  arXiv  Detail & Related papers  (2025-03-06T02:36:59Z)
• Multiview Scene Graph [7.460438046915524]
  A proper scene representation is central to the pursuit of spatial intelligence. We propose to build Multiview Scene Graphs (MSG) from unposed images. MSG represents a scene topologically with interconnected place and object nodes.
  arXiv  Detail & Related papers  (2024-10-15T02:04:05Z)
• Towards Localizing Structural Elements: Merging Geometrical Detection with Semantic Verification in RGB-D Data [0.0]
  This paper presents a real-time pipeline for localizing building components, including wall and ground surfaces, by integrating geometric calculations for pure 3D plane detection. It has a parallel multi-thread architecture to precisely estimate poses and equations of all the planes detected in the environment, filters the ones forming the map structure using a panoptic segmentation validation, and keeps only the validated building components. It can also ensure (re-)association of these detected components into a unified 3D scene graph, bridging the gap between geometric accuracy and semantic understanding.
  arXiv  Detail & Related papers  (2024-09-10T16:28:09Z)
• MeshVPR: Citywide Visual Place Recognition Using 3D Meshes [18.168206222895282]
  Mesh-based scene representation offers a promising direction for simplifying large-scale hierarchical visual localization pipelines. While existing work demonstrates the viability of meshes for visual localization, the impact of using synthetic databases rendered from them in visual place recognition remains largely unexplored. We propose MeshVPR, a novel VPR pipeline that utilizes a lightweight features alignment framework to bridge the gap between real-world and synthetic domains.
  arXiv  Detail & Related papers  (2024-06-04T20:45:53Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)
• SGAligner : 3D Scene Alignment with Scene Graphs [84.01002998166145]
  Building 3D scene graphs has emerged as a topic in scene representation for several embodied AI applications. We focus on the fundamental problem of aligning pairs of 3D scene graphs whose overlap can range from zero to partial. We propose SGAligner, the first method for aligning pairs of 3D scene graphs that is robust to in-the-wild scenarios.
  arXiv  Detail & Related papers  (2023-04-28T14:39:22Z)
• Self-supervised Graph-level Representation Learning with Local and Global Structure [71.45196938842608]
  We propose a unified framework called Local-instance and Global-semantic Learning (GraphLoG) for self-supervised whole-graph representation learning. Besides preserving the local similarities, GraphLoG introduces the hierarchical prototypes to capture the global semantic clusters. An efficient online expectation-maximization (EM) algorithm is further developed for learning the model.
  arXiv  Detail & Related papers  (2021-06-08T05:25:38Z)
• GINet: Graph Interaction Network for Scene Parsing [58.394591509215005]
  We propose a Graph Interaction unit (GI unit) and a Semantic Context Loss (SC-loss) to promote context reasoning over image regions. The proposed GINet outperforms the state-of-the-art approaches on the popular benchmarks, including Pascal-Context and COCO Stuff.
  arXiv  Detail & Related papers  (2020-09-14T02:52:45Z)
• Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions [94.17683799712397]
  We focus on scene graphs, a data structure that organizes the entities of a scene in a graph. We propose a learned method that regresses a scene graph from the point cloud of a scene. We show the application of our method in a domain-agnostic retrieval task, where graphs serve as an intermediate representation for 3D-3D and 2D-3D matching.
  arXiv  Detail & Related papers  (2020-04-08T12:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.