Contextual Graph Representations for Task-Driven 3D Perception and Planning

• URL: http://arxiv.org/abs/2603.26685v1
• Date: Thu, 12 Mar 2026 00:45:12 GMT
• Title: Contextual Graph Representations for Task-Driven 3D Perception and Planning
• Abstract summary: Recent advances in computer vision facilitate fully automatic extraction of object-centric relational representations from visual-inertial data.<n>These state representations, dubbed 3D scene graphs, are a hierarchical decomposition of real-world scenes with a dense multiplex graph structure.<n>This thesis tests the suitability of existing embodied AI environments for research at the intersection of robot task planning and 3D scene graphs.
• Score: 1.7421265706896534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in computer vision facilitate fully automatic extraction of object-centric relational representations from visual-inertial data. These state representations, dubbed 3D scene graphs, are a hierarchical decomposition of real-world scenes with a dense multiplex graph structure. While 3D scene graphs claim to promote efficient task planning for robot systems, they contain numerous objects and relations when only small subsets are required for a given task. This magnifies the state space that task planners must operate over and prohibits deployment in resource constrained settings. This thesis tests the suitability of existing embodied AI environments for research at the intersection of robot task planning and 3D scene graphs and constructs a benchmark for empirical comparison of state-of-the-art classical planners. Furthermore, we explore the use of graph neural networks to harness invariances in the relational structure of planning domains and learn representations that afford faster planning.

Related papers

• An Information-Space Perspective to Scene Graph Sufficiency for Robotic Task Planning [0.45119235878273006]
  Planning in complex environments requires task specifications grounded in representations that capture objects, relations, and affordances.<n>Scene graphs meet this need, but their size in large environments hinders efficient planning.<n>This paper provides a task-centric formalization of what constitutes a sufficient scene graph for planning.
  arXiv  Detail & Related papers  (2026-09-14T13:55:41Z)
• Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces [116.57196064763924]
  We extend the benchmark coverage by introducing dense tabletop objects and explicit multi-level functional relationships.<n>This work introduces critical challenges involving small-scale, dense, and similar instances.<n>We propose an open-vocabulary pipeline based on 2D visual grounding and 3D graph optimization.
  arXiv  Detail & Related papers  (2026-05-15T09:14:50Z)
• Queryable 3D Scene Representation: A Multi-Modal Framework for Semantic Reasoning and Robotic Task Planning [28.803789915686398]
  3D Queryable Scene Representation (3D QSR) is a framework built on multimedia data that unifies three complementary 3D representations.<n>Built on an object-centric design, the framework integrates with large vision-language models to enable semantic queryability.<n>Results demonstrate the framework's ability to facilitate scene understanding and integrate spatial and semantic reasoning.
  arXiv  Detail & Related papers  (2025-09-24T12:53:32Z)
• Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs [44.52978937479273]
  We introduce a multi-robot system that integrates mapping, localization, and task and motion planning (TAMP)<n>Our system builds a shared 3D scene graph incorporating an open-set object-based map, which is leveraged for multi-robot 3D scene graph fusion.<n>We provide an experimental assessment of the performance of our system on real-world tasks in large-scale, outdoor environments.
  arXiv  Detail & Related papers  (2025-06-09T06:02:34Z)
• Exploring 3D Reasoning-Driven Planning: From Implicit Human Intentions to Route-Aware Activity Planning [103.24305074625106]
  We propose 3D Reasoning-Driven Planning, a novel 3D task that reasons the intended activities from implicit instructions and decomposes them into steps with inter-step routes and planning.<n>First, we construct ReasonPlan3D, a large-scale benchmark that covers diverse 3D scenes with rich implicit instructions.<n>Second, we design a novel framework that introduces progressive plan generation with contextual consistency across multiple steps.
  arXiv  Detail & Related papers  (2025-03-17T09:33:58Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning [125.90002884194838]
  ConceptGraphs is an open-vocabulary graph-structured representation for 3D scenes. It is built by leveraging 2D foundation models and fusing their output to 3D by multi-view association. We demonstrate the utility of this representation through a number of downstream planning tasks.
  arXiv  Detail & Related papers  (2023-09-28T17:53:38Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)
• Sequential Manipulation Planning on Scene Graph [90.28117916077073]
  We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential task planning. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with an optimization method. A task plan is then succinct by computing the Graph Editing Distance (GED) between the initial contact graphs and the goal configurations, which generates graph edit operations corresponding to possible robot actions.
  arXiv  Detail & Related papers  (2022-07-10T02:01:33Z)
• Situational Graphs for Robot Navigation in Structured Indoor Environments [9.13466172688693]
  We present a real-time online built Situational Graphs (S-Graphs) composed of a single graph representing the environment. Our method utilizes odometry readings and planar surfaces extracted from 3D LiDAR scans, to construct and optimize in real-time a three layered S-Graph. Our proposal does not only demonstrate state-of-the-art results for pose estimation of the robot, but also contributes with a metric-semantic-topological model of the environment.
  arXiv  Detail & Related papers  (2022-02-24T16:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.