A Scene Graph Backed Approach to Open Set Semantic Mapping

• URL: http://arxiv.org/abs/2602.03781v1
• Date: Tue, 03 Feb 2026 17:41:51 GMT
• Title: A Scene Graph Backed Approach to Open Set Semantic Mapping
• Abstract summary: Open Set Semantic Mapping and 3D Scene Graphs (3DSSGs) are established paradigms in robotic perception.<n>We propose a mapping architecture where the 3DSSG serves as the foundational backend, acting as the primary knowledge representation.<n>This ensures that the map remains topologically consistent and computationally efficient, even during extended operations in large-scale settings.
• Score: 1.8266902367595235
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Open Set Semantic Mapping and 3D Semantic Scene Graphs (3DSSGs) are established paradigms in robotic perception, deploying them effectively to support high-level reasoning in large-scale, real-world environments remains a significant challenge. Most existing approaches decouple perception from representation, treating the scene graph as a derivative layer generated post hoc. This limits both consistency and scalability. In contrast, we propose a mapping architecture where the 3DSSG serves as the foundational backend, acting as the primary knowledge representation for the entire mapping process. Our approach leverages prior work on incremental scene graph prediction to infer and update the graph structure in real-time as the environment is explored. This ensures that the map remains topologically consistent and computationally efficient, even during extended operations in large-scale settings. By maintaining an explicit, spatially grounded representation that supports both flat and hierarchical topologies, we bridge the gap between sub-symbolic raw sensor data and high-level symbolic reasoning. Consequently, this provides a stable, verifiable structure that knowledge-driven frameworks, ranging from knowledge graphs and ontologies to Large Language Models (LLMs), can directly exploit, enabling agents to operate with enhanced interpretability, trustworthiness, and alignment to human concepts.

Related papers

• Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments [20.027960785504586]
  We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation as a probabilistic alignment problem.<n>We validate this approach across diverse simulated residential datasets and large, open-plan real-world environments.
  arXiv  Detail & Related papers  (2026-08-06T15:36:12Z)
• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces [116.57196064763924]
  We extend the benchmark coverage by introducing dense tabletop objects and explicit multi-level functional relationships.<n>This work introduces critical challenges involving small-scale, dense, and similar instances.<n>We propose an open-vocabulary pipeline based on 2D visual grounding and 3D graph optimization.
  arXiv  Detail & Related papers  (2026-05-15T09:14:50Z)
• ToLL: Topological Layout Learning with Structural Multi-view Augmentation for 3D Scene Graph Pretraining [29.799719863247287]
  3D Scene Graph (3DSG) generation plays a pivotal role in spatial understanding and semantic-affordance perception.<n>Current solutions primarily focus on cross-modal assisted representation learning and object-centric generation pre-training.<n>We propose a Topological Layout Learning (ToLL) for 3DSG pretraining framework.<n>In detail, we design an Anchor-Conditioned Topological Geometry Reasoning, with a GNN to recover the global layout of zero-centered subgraphs by the spatial priors from sparse anchors.
  arXiv  Detail & Related papers  (2026-03-30T08:43:52Z)
• OGScene3D: Incremental Open-Vocabulary 3D Gaussian Scene Graph Mapping for Scene Understanding [30.730672505996967]
  OGScene3D is an open-vocabulary scene understanding system that achieves accurate 3D semantic mapping and scene graph construction incrementally.<n>Our system employs a confidence-based Gaussian semantic representation that jointly models semantic predictions and their reliability.<n>By integrating 2D-3D semantic consistency with Gaussian rendering contribution, this method continuously refines the semantic understanding of the entire scene.
  arXiv  Detail & Related papers  (2026-03-17T09:38:01Z)
• SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D [51.32219731589742]
  3D scene graphs provide a structured representation of object entities and their relationships.<n>Existing approaches for 3D scene graph generation typically combine scene reconstruction with graph neural networks (GNNs)<n>In this work, we introduce a Scene Graph Retrieval-Reasoning Model in 3D (SGR3 Model)
  arXiv  Detail & Related papers  (2026-03-04T21:19:54Z)
• Relationship-Aware Hierarchical 3D Scene Graph for Task Reasoning [10.364779390403337]
  We propose an enhanced hierarchical 3D scene graph that integrates open-vocabulary features across multiple abstraction levels and supports object-relational reasoning.<n>We introduce a task reasoning module that combines Large Language Models (LLM) and a Vision Language Model (VLM) to interpret the scene graph's semantic and relational information.<n>We validate our method by deploying it on a quadruped robot in multiple environments and tasks, highlighting its ability to reason about them.
  arXiv  Detail & Related papers  (2026-02-02T18:47:02Z)
• KeySG: Hierarchical Keyframe-Based 3D Scene Graphs [1.5134439544218246]
  KeySG represents 3D scenes as a hierarchical graph consisting of floors, rooms, objects, and functional elements.<n>We leverage VLM to extract scene information, alleviating the need to explicitly model relationship edges between objects.<n>Our approach can process complex and ambiguous queries while mitigating the scalability issues associated with large scene graphs.
  arXiv  Detail & Related papers  (2025-10-01T15:53:27Z)
• Graph-Guided Dual-Level Augmentation for 3D Scene Segmentation [21.553363236403822]
  3D point cloud segmentation aims to assign semantic labels to individual points in a scene for fine-grained spatial understanding.<n>Existing methods typically adopt data augmentation to alleviate the burden of large-scale annotation.<n>We propose a graph-guided data augmentation framework with dual-level constraints for realistic 3D scene synthesis.
  arXiv  Detail & Related papers  (2025-07-30T13:25:36Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• On Support Relations Inference and Scene Hierarchy Graph Construction from Point Cloud in Clustered Environments [3.4535508414601344]
  In 3D scenes, rich spatial geometric and topological information are often ignored by RGB-based approaches for scene understanding. In this study, we develop a bottom-up approach for scene understanding that infers support relations between objects from a point cloud.
  arXiv  Detail & Related papers  (2024-04-22T02:42:32Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)
• GINet: Graph Interaction Network for Scene Parsing [58.394591509215005]
  We propose a Graph Interaction unit (GI unit) and a Semantic Context Loss (SC-loss) to promote context reasoning over image regions. The proposed GINet outperforms the state-of-the-art approaches on the popular benchmarks, including Pascal-Context and COCO Stuff.
  arXiv  Detail & Related papers  (2020-09-14T02:52:45Z)
• Spatial Pyramid Based Graph Reasoning for Semantic Segmentation [67.47159595239798]
  We apply graph convolution into the semantic segmentation task and propose an improved Laplacian. The graph reasoning is directly performed in the original feature space organized as a spatial pyramid. We achieve comparable performance with advantages in computational and memory overhead.
  arXiv  Detail & Related papers  (2020-03-23T12:28:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.