Open-Set 3D Scene Graphs for Field Robotics: An Outdoor Case Study

• URL: http://arxiv.org/abs/2609.04607v1
• Date: Fri, 04 Sep 2026 01:14:09 GMT
• Title: Open-Set 3D Scene Graphs for Field Robotics: An Outdoor Case Study
• Abstract summary: Three-dimensional scene graphs (3DSGs) have emerged as a promising approach for building geometrically grounded, semantically informed maps.<n>We analyze the components common to most 3DSG representations across five outdoor robotic datasets to characterize challenges that arise in complex outdoor environments.<n>Our results show that outdoor 3DSGs can maintain compact (less than $600$MB for multi-kilometer trajectories) and relatively consistent large-scale environment representations.
• Score: 5.807679246334387
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Three-dimensional scene graphs (3DSGs) have emerged as a promising approach for building geometrically grounded, semantically informed, hierarchical general-purpose maps to support high-level robotic reasoning. However, the behavior of 3DSGs in real-world outdoor deployments remains poorly understood, particularly when combined with open-set vision-language models (VLMs). In this field report, we analyze the components common to most 3DSG representations across five outdoor robotic datasets to characterize challenges that arise in complex outdoor environments. Using the recently proposed Terra 3DSG as a case study, we investigate semantic point embeddings, place-node graph navigation, region-level understanding, and memory size across the five diverse datasets. We additionally introduce novel consistency metrics to evaluate whether semantic and structural graph properties remain stable across repeated traversals of the same environment. Our analysis reveals that outliers and multiple modes are common in VLM point embeddings across all tested datasets with outlier ratios above $0.1$ for around $30\%$ of points. We demonstrate the feasibility of outdoor 3DSGs for navigation-based object retrieval, achieving success rates near $70\%$, though performance is limited by traversability failures and inefficient routing, with trajectories averaging approximately $66\%$ suboptimal path efficiency. Region-level understanding remains challenging in complex natural environments, with low average F1 scores around $0.359$. Overall, our results show that outdoor 3DSGs can maintain compact (less than $600$MB for multi-kilometer trajectories) and relatively consistent large-scale environment representations, while highlighting open challenges in handling multiple semantic modes, incorporating traversability into graph structures, and improving higher-level region understanding.

Related papers

• 3D Scene Graphs: Open Challenges and Future Directions [56.932031090019855]
  3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment.<n>This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions.
  arXiv  Detail & Related papers  (2026-06-15T08:14:08Z)
• Hierarchical Object Representation for Spatial Robot Perception: Points, Meshes, and Superquadrics [7.735876865294663]
  Hierarchical 3D Scene Graphs (3DSG) have emerged as an actionable and scalable representation for long-term autonomy.<n>We introduce a hierarchical object representation that can be leveraged for high-fidelity object-level reconstruction.<n>We show that our superquadric-based map alignment method outperforms the current state-of-the-art object based map alignment method ROMAN.
  arXiv  Detail & Related papers  (2026-06-01T01:45:47Z)
• 3D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation [48.641376758484505]
  Vision-language navigation (VLN) requires an agent to traverse complex 3D environments based on natural language instructions.<n>This work proposes a 3D Gaussian Map that represents the environment as a set of differentiable 3D Gaussians.<n>Egocentric Scene Map is constructed online by initializing 3D Gaussians from sparse pseudo-lidar point clouds.
  arXiv  Detail & Related papers  (2026-05-26T03:25:49Z)
• Geo3DVQA: Evaluating Vision-Language Models for 3D Geospatial Reasoning from Aerial Imagery [18.7420518276348]
  Geo3DVQA is a benchmark for evaluating vision-language models (VLMs) in height-aware, 3D geospatial reasoning.<n>Unlike conventional sensor-based frameworks, Geo3DVQA emphasizes realistic scenarios that integrate elevation, sky view factors, and land cover patterns.
  arXiv  Detail & Related papers  (2025-12-08T08:16:14Z)
• 3dSAGER: Geospatial Entity Resolution over 3D Objects (Technical Report) [7.378893412842889]
  3dSAGER is an end-to-end pipeline for geospatial entity resolution over 3D objects.<n>We present a novel, spatial-reference-independent featurization mechanism that captures intricate geometric characteristics of matching pairs.<n>We also propose a new lightweight and interpretable blocking method, BKAFI, that leverages a trained model to efficiently generate high-recall candidate sets.
  arXiv  Detail & Related papers  (2025-11-09T09:35:45Z)
• SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting [104.83629308412958]
  3D Gaussian Splatting (3DGS) serves as a highly performant and efficient encoding of scene geometry, appearance, and semantics.<n>We propose the first large-scale benchmark that systematically assesses three groups of methods directly in 3D space.<n>Results demonstrate a clear advantage of the generalizable paradigm, particularly in relaxing the scene-specific limitation.
  arXiv  Detail & Related papers  (2025-06-10T11:52:45Z)
• MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations [55.022519020409405]
  This paper builds the first largest ever multi-modal 3D scene dataset and benchmark with hierarchical grounded language annotations, MMScan.<n>The resulting multi-modal 3D dataset encompasses 1.4M meta-annotated captions on 109k objects and 7.7k regions as well as over 3.04M diverse samples for 3D visual grounding and question-answering benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T17:59:30Z)
• Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata [70.9375320609781]
  We aim to generate fine-grained 3D geometry from large-scale sparse LiDAR scans, abundantly captured by autonomous vehicles (AV) We propose hierarchical Generative Cellular Automata (hGCA), a spatially scalable 3D generative model, which grows geometry with local kernels following, in a coarse-to-fine manner, equipped with a light-weight planner to induce global consistency.
  arXiv  Detail & Related papers  (2024-06-12T14:56:56Z)
• Uni3DETR: Unified 3D Detection Transformer [75.35012428550135]
  We propose a unified 3D detector that addresses indoor and outdoor detection within the same framework. Specifically, we employ the detection transformer with point-voxel interaction for object prediction. We then propose the mixture of query points, which sufficiently exploits global information for dense small-range indoor scenes and local information for large-range sparse outdoor ones.
  arXiv  Detail & Related papers  (2023-10-09T13:20:20Z)
• OpenGF: An Ultra-Large-Scale Ground Filtering Dataset Built Upon Open ALS Point Clouds Around the World [22.577705573226257]
  We present OpenGF, first Ultra-Large-Scale Ground Filtering dataset. OpenGF contains more than half a billion finely labeled ground and non-ground points. We evaluate the performance of state-of-the-art rule-based algorithms and 3D semantic segmentation networks on our dataset.
  arXiv  Detail & Related papers  (2021-01-24T04:07:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.