OGScene3D: Incremental Open-Vocabulary 3D Gaussian Scene Graph Mapping for Scene Understanding

• URL: http://arxiv.org/abs/2603.16301v2
• Date: Wed, 18 Mar 2026 02:47:56 GMT
• Title: OGScene3D: Incremental Open-Vocabulary 3D Gaussian Scene Graph Mapping for Scene Understanding
• Abstract summary: OGScene3D is an open-vocabulary scene understanding system that achieves accurate 3D semantic mapping and scene graph construction incrementally.<n>Our system employs a confidence-based Gaussian semantic representation that jointly models semantic predictions and their reliability.<n>By integrating 2D-3D semantic consistency with Gaussian rendering contribution, this method continuously refines the semantic understanding of the entire scene.
• Score: 30.730672505996967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-vocabulary scene understanding is crucial for robotic applications, enabling robots to comprehend complex 3D environmental contexts and supporting various downstream tasks such as navigation and manipulation. However, existing methods require pre-built complete 3D semantic maps to construct scene graphs for scene understanding, which limits their applicability in robotic scenarios where environments are explored incrementally. To address this challenge, we propose OGScene3D, an open-vocabulary scene understanding system that achieves accurate 3D semantic mapping and scene graph construction incrementally. Our system employs a confidence-based Gaussian semantic representation that jointly models semantic predictions and their reliability, enabling robust scene modeling. Building on this representation, we introduce a hierarchical 3D semantic optimization strategy that achieves semantic consistency through local correspondence establishment and global refinement, thereby constructing globally consistent semantic maps. Moreover, we design a long-term global optimization method that leverages temporal memory of historical observations to enhance semantic predictions. By integrating 2D-3D semantic consistency with Gaussian rendering contribution, this method continuously refines the semantic understanding of the entire scene. Furthermore, we develop a progressive graph construction approach that dynamically creates and updates both nodes and semantic relationships, allowing continuous updating of the 3D scene graphs. Extensive experiments on widely used datasets and real-world scenes demonstrate the effectiveness of our OGScene3D on open-vocabulary scene understanding.

Related papers

• From Pixels to Concepts: Growing Rich 3D Semantic Scene Graph Forests utilizing Foundation Models [4.137761255401348]
  Hierarchical 3D scene graphs integrate geometric, semantic, and relational data within a unified spatial framework.<n>Current 3D scene graph approaches often restrict themselves to rigid structures of pre-determined relationship classes.<n>This paper explores the potential of foundation models to build forests of 3D scene graphs with open semantic relationships.
  arXiv  Detail & Related papers  (2026-06-22T13:26:17Z)
• EPS3D: End-to-End Feed-Forward 3D Panoptic Segmentation [92.9295124221522]
  EPS3D is an end-to-end feed-forward framework for open-vocabulary 3D panoptic segmentation.<n>It predicts semantic and instance features from multi-view images, improving 3D consistency and avoiding error accumulation.<n>It supports tasks like robotic manipulation and 3D scene editing.
  arXiv  Detail & Related papers  (2026-06-08T03:26:54Z)
• DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding [6.961193964556352]
  We present DGSG-Mind, a hybrid instance-aware 3D Gaussian dynamic scene graph system with an embodied reasoning agent.<n>We show that DGSG-Mind achieves the best zero-shot 3DVG performance among methods operating on self-reconstructed maps.<n>We also deploy DGSG-Mind on real-world robots to demonstrate its target-oriented reasoning and dynamic update capabilities.
  arXiv  Detail & Related papers  (2026-05-28T13:04:17Z)
• Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation [5.189354278537469]
  GuidedSceneGen produces metrically accurate, globally consistent, and semantically interpretable indoor scenes.<n>Our approach maintains an absolute world coordinate frame throughout the entire generation process.
  arXiv  Detail & Related papers  (2026-03-14T11:46:27Z)
• A Scene Graph Backed Approach to Open Set Semantic Mapping [1.8266902367595235]
  Open Set Semantic Mapping and 3D Scene Graphs (3DSSGs) are established paradigms in robotic perception.<n>We propose a mapping architecture where the 3DSSG serves as the foundational backend, acting as the primary knowledge representation.<n>This ensures that the map remains topologically consistent and computationally efficient, even during extended operations in large-scale settings.
  arXiv  Detail & Related papers  (2026-02-03T17:41:51Z)
• Open-World 3D Scene Graph Generation for Retrieval-Augmented Reasoning [24.17324180628543]
  We propose a unified framework for Open-World 3D Scene Graph Generation with Retrieval-Augmented Reasoning.<n>Our method integrates Vision-Language Models (VLMs) with retrieval-based reasoning to support multimodal exploration and language-guided interaction.<n>We evaluate our method on 3DSSG and Replica benchmarks across four tasks-scene question answering, visual grounding, instance retrieval, and task planning-demonstrating robust generalization and superior performance in diverse environments.
  arXiv  Detail & Related papers  (2025-11-08T07:37:29Z)
• A Study of the Framework and Real-World Applications of Language Embedding for 3D Scene Understanding [2.934809709666499]
  Gaussian Splatting has emerged as a transformative technique for real-time 3D scene representation.<n>Its ability to render complex scenes with high fidelity has enabled progress across domains such as scene reconstruction, robotics, and interactive content creation.<n>This survey presents a structured review of current research efforts that combine language guidance with 3D Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-08-07T06:33:08Z)
• Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphs [16.153129392697885]
  We introduce a training-free framework that constructs a superpoint graph directly from Gaussian primitives.<n>The superpoint graph partitions the scene into spatially compact and semantically coherent regions, forming view-consistent 3D entities.<n>Our method achieves state-of-the-art open-vocabulary segmentation performance, with semantic field reconstruction completed over $30times$ faster.
  arXiv  Detail & Related papers  (2025-04-17T17:56:07Z)
• EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understanding [72.96388875744704]
  3D occupancy prediction provides a comprehensive description of the surrounding scenes.<n>Most existing methods focus on offline perception from one or a few views.<n>We formulate an embodied 3D occupancy prediction task to target this practical scenario and propose a Gaussian-based EmbodiedOcc framework to accomplish it.
  arXiv  Detail & Related papers  (2024-12-05T17:57:09Z)
• Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding [59.51535163599723]
  FreeGS is an unsupervised semantic-embedded 3DGS framework that achieves view-consistent 3D scene understanding without the need for 2D labels.<n>FreeGS performs comparably to state-of-the-art methods while avoiding the complex data preprocessing workload.
  arXiv  Detail & Related papers  (2024-11-29T08:52:32Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting [53.6394928681237]
  holistic understanding of urban scenes based on RGB images is a challenging yet important problem. Our main idea involves the joint optimization of geometry, appearance, semantics, and motion using a combination of static and dynamic 3D Gaussians. Our approach offers the ability to render new viewpoints in real-time, yielding 2D and 3D semantic information with high accuracy.
  arXiv  Detail & Related papers  (2024-03-19T13:39:05Z)
• FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding [11.118857208538039]
  We present Foundation Model Embedded Gaussian Splatting (S), which incorporates vision-language embeddings of foundation models into 3D Gaussian Splatting (GS) Results demonstrate remarkable multi-view semantic consistency, facilitating diverse downstream tasks, beating state-of-the-art methods by 10.2 percent on open-vocabulary language-based object detection. This research explores the intersection of vision, language, and 3D scene representation, paving the way for enhanced scene understanding in uncontrolled real-world environments.
  arXiv  Detail & Related papers  (2024-01-03T20:39:02Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)
• SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion [86.77318031029404]
  We propose a framework that performs scene reconstruction and semantic scene completion jointly in an incremental and real-time manner. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model.
  arXiv  Detail & Related papers  (2020-10-26T15:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.