Generation of High-Level Concepts in 3D Scene Graphs via Autoregressive Diffusion

• URL: http://arxiv.org/abs/2608.28733v1
• Date: Fri, 28 Aug 2026 17:41:37 GMT
• Title: Generation of High-Level Concepts in 3D Scene Graphs via Autoregressive Diffusion
• Abstract summary: 3DSGs represent environments as multi-layer hierarchies that connect observed geometric primitives to metric-semantic concepts.<n>We propose a unified autoregressive diffusion-based graph generative model that jointly learns structure and features.<n>Our method consistently surpasses all learning-based and random baselines across 3DSG datasets spanning synthetic scenes, real architectural floor plans, and robotic sensor data.
• Score: 4.721534404221582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Indoor 3D Scene Graphs (3DSGs) represent environments as multi-layer hierarchies that connect observed geometric primitives (e.g., planes) to higher-level metric-semantic concepts (e.g., rooms, floors, buildings), enabling incremental spatial reasoning for robotic perception and SLAM. However, classical high-level concept generation approaches rely on hand-crafted rules for specific concept classes, while learning-based methods require separate models for graph structure and spatial node features (e.g., centroids), which limits scalability to novel classes and more complex hierarchies. We propose a unified autoregressive diffusion-based graph generative model that jointly learns structure and features, constructing complete 3DSGs bottom-up from observed vertical planes across arbitrary hierarchy depths. Our method consistently surpasses all learning-based and random baselines across 3DSG datasets spanning synthetic scenes, real architectural floor plans, and robotic sensor data, with varying layout complexity and hierarchy depth, and surpasses a one-shot model with oracle access to the target graph size on the largest hierarchy and on real single-floor data. Finally, we propose an adaptation of the Fused Gromov--Wasserstein distance for principled graph-level evaluation of generated 3DSGs against ground truth.

Related papers

• Hierarchical Object Representation for Spatial Robot Perception: Points, Meshes, and Superquadrics [7.735876865294663]
  Hierarchical 3D Scene Graphs (3DSG) have emerged as an actionable and scalable representation for long-term autonomy.<n>We introduce a hierarchical object representation that can be leveraged for high-fidelity object-level reconstruction.<n>We show that our superquadric-based map alignment method outperforms the current state-of-the-art object based map alignment method ROMAN.
  arXiv  Detail & Related papers  (2026-06-01T01:45:47Z)
• LoST: Level of Semantics Tokenization for 3D Shapes [50.847769883816085]
  State-of-the-art methods rely on geometric level-of-detail (LoD) hierarchies, originally designed for rendering and compression.<n>We propose Level-of-Semantics Tokenization (LoST), which orders salience, such that early prefixes decode complete, plausible shapes.<n>LoST achieves SOTA reconstruction, surpassing previous LoD-based 3D shape tokenizers by large margins on both geometric and semantic reconstruction metrics.
  arXiv  Detail & Related papers  (2026-03-18T17:56:06Z)
• Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video [76.32954467706581]
  We propose SAGE, a framework for Scalable Adaptation of GEometric foundation models from raw video streams.<n>We use a hierarchical mining pipeline to transform videos into training trajectories and hybrid supervision.<n>Experiments show that SAGE significantly enhances zero-shot generalization, reducing Chamfer Distance by 20-42% on unseen benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T09:53:21Z)
• A Scene Graph Backed Approach to Open Set Semantic Mapping [1.8266902367595235]
  Open Set Semantic Mapping and 3D Scene Graphs (3DSSGs) are established paradigms in robotic perception.<n>We propose a mapping architecture where the 3DSSG serves as the foundational backend, acting as the primary knowledge representation.<n>This ensures that the map remains topologically consistent and computationally efficient, even during extended operations in large-scale settings.
  arXiv  Detail & Related papers  (2026-02-03T17:41:51Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• WorldGrow: Generating Infinite 3D World [75.81531067447203]
  We tackle the challenge of generating the infinitely extendable 3D world -- large, continuous environments with coherent geometry and realistic appearance.<n>We propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis.<n>Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity.
  arXiv  Detail & Related papers  (2025-10-24T17:39:52Z)
• SeqAffordSplat: Scene-level Sequential Affordance Reasoning on 3D Gaussian Splatting [85.87902260102652]
  We introduce the novel task of Sequential 3D Gaussian Affordance Reasoning.<n>We then propose SeqSplatNet, an end-to-end framework that directly maps an instruction to a sequence of 3D affordance masks.<n>Our method sets a new state-of-the-art on our challenging benchmark, effectively advancing affordance reasoning from single-step interactions to complex, sequential tasks at the scene level.
  arXiv  Detail & Related papers  (2025-07-31T17:56:55Z)
• Open-Vocabulary Indoor Object Grounding with 3D Hierarchical Scene Graph [0.0]
  OVIGo-3DHSG represents an extensive indoor environment over a Hierarchical Scene Graph.<n>The hierarchical representation explicitly models spatial relations across floors, rooms, locations, and objects.<n>Our approach demonstrates efficient scene comprehension and robust object grounding compared to existing methods.
  arXiv  Detail & Related papers  (2025-07-16T10:47:12Z)
• Intelligent Spatial Perception by Building Hierarchical 3D Scene Graphs for Indoor Scenarios with the Help of LLMs [4.764379183672723]
  This paper introduces a novel system that harnesses the capabilities of Large Language Models (LLMs) to construct hierarchical 3D Scene Graphs for indoor scenarios.<n>The proposed framework constructs 3DSGs consisting of a fundamental layer with rich metric-semantic information, an object layer featuring precise point-cloud representation of object nodes, and higher layers of room, floor, and building nodes.<n>Thanks to the innovative application of LLMs, not only object nodes but also nodes of higher layers, e.g., room nodes, are annotated in an intelligent and accurate manner.
  arXiv  Detail & Related papers  (2025-03-19T10:40:28Z)
• Hierarchical Graph Networks for 3D Human Pose Estimation [50.600944798627786]
  Recent 2D-to-3D human pose estimation works tend to utilize the graph structure formed by the topology of the human skeleton. We argue that this skeletal topology is too sparse to reflect the body structure and suffer from serious 2D-to-3D ambiguity problem. We propose a novel graph convolution network architecture, Hierarchical Graph Networks, to overcome these weaknesses.
  arXiv  Detail & Related papers  (2021-11-23T15:09:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.