3D Scene Graphs: Open Challenges and Future Directions

• URL: http://arxiv.org/abs/2606.19383v1
• Date: Mon, 15 Jun 2026 08:14:08 GMT
• Title: 3D Scene Graphs: Open Challenges and Future Directions
• Abstract summary: 3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment.<n>This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions.
• Score: 56.932031090019855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including manipulation, navigation, task planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real-world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are built from raw sensory observations, discussing the most common terminologies, conventions, and techniques. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed content, accessible at https://3dscenegraphs.com/.

Related papers

• Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis [59.527737790821305]
  We propose a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding.<n>Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships.<n>Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability.
  arXiv  Detail & Related papers  (2026-07-06T17:29:49Z)
• OP3DSG: Open-Vocabulary Part-Aware 3D Scene Graph Generation for Real-World Environments [4.039245878626346]
  3D scene graphs (3DSGs) provide a compact and structured abstraction of 3D environments.<n>We propose OP3DSG, an open-vocabulary part-aware 3DSG generation framework.<n> OP3DSG integrates object-part knowledge-guided detection with part-aware 3D fusion to preserve small and interaction-relevant components.
  arXiv  Detail & Related papers  (2026-06-29T05:05:54Z)
• DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding [6.961193964556352]
  We present DGSG-Mind, a hybrid instance-aware 3D Gaussian dynamic scene graph system with an embodied reasoning agent.<n>We show that DGSG-Mind achieves the best zero-shot 3DVG performance among methods operating on self-reconstructed maps.<n>We also deploy DGSG-Mind on real-world robots to demonstrate its target-oriented reasoning and dynamic update capabilities.
  arXiv  Detail & Related papers  (2026-05-28T13:04:17Z)
• A Systematic Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation [2.3895981099137535]
  Point cloud stands as the most widely adopted format for representing 3D shapes and scenes due to its simplicity and geometric fidelity.<n>Its inherent unordered and irregular nature, exacerbated by sensor noise and occlusions, introduces unique challenges for machine learning based methodologies.<n>Our focus is directed towards deep learning models for three fundamental tasks in 3D vision: point cloud classification, part segmentation, and semantic segmentation.
  arXiv  Detail & Related papers  (2026-05-16T19:37:41Z)
• Open-World 3D Scene Graph Generation for Retrieval-Augmented Reasoning [24.17324180628543]
  We propose a unified framework for Open-World 3D Scene Graph Generation with Retrieval-Augmented Reasoning.<n>Our method integrates Vision-Language Models (VLMs) with retrieval-based reasoning to support multimodal exploration and language-guided interaction.<n>We evaluate our method on 3DSSG and Replica benchmarks across four tasks-scene question answering, visual grounding, instance retrieval, and task planning-demonstrating robust generalization and superior performance in diverse environments.
  arXiv  Detail & Related papers  (2025-11-08T07:37:29Z)
• A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation [66.62489208150681]
  3D Gaussian Splatting (3DGS) has emerged as a powerful alternative to Neural Radiance Fields (NeRF) for 3D scene representation.<n>This survey provides a comprehensive overview of recent progress in 3DGS applications.
  arXiv  Detail & Related papers  (2025-08-13T17:44:39Z)
• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data [89.77871049500546]
  We present a unified framework for enhancing 3D spatial reasoning in pre-trained vision-language models without modifying their architecture.<n>This framework combines SpatialMind, a structured prompting strategy that decomposes complex scenes and questions into interpretable reasoning steps, with ScanForgeQA, a scalable question-answering dataset built from diverse 3D simulation scenes.
  arXiv  Detail & Related papers  (2025-06-04T07:36:33Z)
• Agentic 3D Scene Generation with Spatially Contextualized VLMs [67.31920821192323]
  We introduce a new paradigm that enables vision-language models to generate, understand, and edit complex 3D environments.<n>We develop an agentic 3D scene generation pipeline in which the VLM iteratively reads from and updates the spatial context.<n>Results show that our framework can handle diverse and challenging inputs, achieving a level of generalization not observed in prior work.
  arXiv  Detail & Related papers  (2025-05-26T15:28:17Z)
• Embodied Intelligence for 3D Understanding: A Survey on 3D Scene Question Answering [28.717312557697376]
  3D Scene Question Answering represents an interdisciplinary task that integrates 3D visual perception and natural language processing.<n>Recent advances in large multimodal modelling have driven the creation of diverse datasets.<n>We provide the first comprehensive and systematic review of 3D SQA.
  arXiv  Detail & Related papers  (2025-02-01T07:01:33Z)
• GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding [53.42728468191711]
  Open-Vocabulary 3D object affordance grounding aims to anticipate action possibilities'' regions on 3D objects with arbitrary instructions.<n>We propose GREAT (GeometRy-intEntion collAboraTive inference) for Open-Vocabulary 3D Object Affordance Grounding.
  arXiv  Detail & Related papers  (2024-11-29T11:23:15Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• A Survey on 3D Gaussian Splatting [61.50539646390613]
  3D Gaussian splatting (GS) has emerged as a transformative technique in radiance fields.<n>We provide the first systematic overview of the recent developments and critical contributions in 3D GS.<n>By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond.
  arXiv  Detail & Related papers  (2024-01-08T13:42:59Z)
• Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis [21.920148546359016]
  We propose a 3D point-based scene graph generation framework to bridge perception and reasoning. Within the reasoning stage, an EDGE-oriented Graph Convolutional Network is created to exploit multi-dimensional edge features. Our experimental results show promising edge-oriented reasoning effects on scene graph generation studies.
  arXiv  Detail & Related papers  (2021-03-09T17:09:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.