Hierarchical 3D Scene Graph Construction and Belief-based Planning for Semantic Navigation

• URL: http://arxiv.org/abs/2606.31071v1
• Date: Tue, 30 Jun 2026 03:01:44 GMT
• Title: Hierarchical 3D Scene Graph Construction and Belief-based Planning for Semantic Navigation
• Abstract summary: We propose a zero-shot semantic navigation framework for embodied agents.<n>Our method incrementally maintains an online Hierarchical 3D Scene Graph (HSG) over objects, zones, and regions, serving as a compact state abstraction for global planning.<n>Building on this memory, we introduce a hierarchical belief-based planning framework that fuses semantic priors with exploration evidence on the HSG, and performs finite-horizon rollouts on an HSG-based simulator to explicitly estimate the long-term expected returns of candidate macro-actions.
• Score: 34.08516675160383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic navigation is a fundamental task for embodied agents operating in unseen environments, requiring both semantic understanding and long-term decision-making. Recent foundation models have empowered agents with rich semantic priors for this task. However, without structured global representations, decision-making often falls back on local observations and greedy strategies, resulting in inefficient exploration and myopic behaviors, especially in long-distance navigation. To address these challenges, we propose a zero-shot semantic navigation framework. Our method incrementally maintains an online Hierarchical 3D Scene Graph (HSG) to form a multi-granular semantic topology over objects, zones, and regions, serving as a compact state abstraction for global planning. Building on this memory, we introduce a hierarchical belief-based planning framework that fuses semantic priors with exploration evidence on the HSG, and performs finite-horizon rollouts on an HSG-based simulator to explicitly estimate the long-term expected returns of candidate macro-actions. This enables globally consistent decisions and reduces redundant backtracking. Extensive experiments in high-fidelity simulation environments across multiple tasks and datasets demonstrate that our method outperforms existing state-of-the-art methods, particularly in long-distance scenarios, where our approach improves SR and SPL by an average of 9.4\% and 5.0\%, respectively.

Related papers

• Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments [20.027960785504586]
  We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation as a probabilistic alignment problem.<n>We validate this approach across diverse simulated residential datasets and large, open-plan real-world environments.
  arXiv  Detail & Related papers  (2026-08-06T15:36:12Z)
• SAGE-Nav: Leveraging LLM Planning and Alignment Fusion for Hierarchical Scene Graph-Guided Navigation [26.212246536363363]
  Object-Goal Navigation (Nav) requires embodied agents to autonomously locate specified targets using only egocentric visual observations.<n>We propose SAGE-Nav, a novel hierarchical framework that integrates the reasoning capabilities of Large Language Models (LLMs) with dynamic scene graphs.<n>We show that SAGE-Nav achieves substantial gains in navigation efficiency and zero-shot generalization while maintaining the low control latency required for physical robotic deployment.
  arXiv  Detail & Related papers  (2026-06-24T07:24:39Z)
• STEM: Semantic Target Search and Exploration using MAVs in Cluttered Environments [17.009653719709213]
  This paper proposes a novel framework that utilizes a semantically-guided viewpoint planner to minimize target search and exploration time.<n>Specifically, we develop a planner that generates efficient semantic exploration plans by prioritizing viewpoints that likely lead to the target.
  arXiv  Detail & Related papers  (2026-05-30T15:00:58Z)
• GeoMamba: A Geometry-driven MambaVision Framework and Dataset for Fine-grained Optical-SAR Object Retrieval [54.741349848771144]
  GeoMamba is a geometry-driven framework for optical-SAR fine-grained retrieval.<n>GFI module enhances cross-modal feature interaction and incorporates structural priors.<n>GeoMamba outperforms existing methods, achieving 63.3% mAP and 77.0% Rank-1 accuracy in all-to-all retrieval setting.
  arXiv  Detail & Related papers  (2026-05-19T12:08:09Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• SignNav: Leveraging Signage for Semantic Visual Navigation in Large-Scale Indoor Environments [57.79171900005793]
  Humans routinely leverage semantic hints provided by signage to navigate to destinations within Large-Scale Indoor environments.<n>This paper introduces a novel embodied navigation task, SignNav, which requires the agent to interpret semantic hint from signage and reason about the subsequent action based on current observation.<n>We propose the spatial-aware module grounds the semantic hint of signage into physical world, while the temporal-aware module captures long-range dependencies between historical states and current observation.
  arXiv  Detail & Related papers  (2026-03-17T06:36:26Z)
• RAGNav: A Retrieval-Augmented Topological Reasoning Framework for Multi-Goal Visual-Language Navigation [1.7508558850131373]
  Vision-Language Navigation (VLN) is evolving from single-point pathfinding toward the more challenging Multi-Goal VLN.<n>RAGNav is a framework that bridges the gap between semantic reasoning and physical structure.
  arXiv  Detail & Related papers  (2026-03-04T05:31:33Z)
• USS-Nav: Unified Spatio-Semantic Scene Graph for Lightweight UAV Zero-Shot Object Navigation [10.934912400963588]
  USS-Nav is a lightweight framework that incrementally constructs a Unified Spatio-Semantic scene graph.<n>It enables efficient Large Language Model (LLM)-augmented Zero-Shot Object Navigation in unknown environments.<n>Our framework outperforms state-of-the-art methods in terms of computational efficiency and real-time update frequency.
  arXiv  Detail & Related papers  (2026-01-31T13:10:02Z)
• Dynamic Topology Awareness: Breaking the Granularity Rigidity in Vision-Language Navigation [22.876516699004814]
  Vision-Language Navigation in Continuous Environments (VLN-CE) presents a core challenge: grounding high-level linguistic instructions into precise, safe, and long-horizon spatial actions.<n>Explicit topological maps have proven to be a vital solution for providing robust spatial memory in such tasks.<n>Existing topological planning methods suffer from a "Granularity Rigidity" problem.<n>We propose DGNav, a framework for Dynamic Topological Navigation, introducing a context-aware mechanism to modulate map density and connectivity on-the-fly.
  arXiv  Detail & Related papers  (2026-01-29T14:06:23Z)
• RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning [61.84363374647606]
  Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions.<n>These descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning.<n>We propose a reasoning-guided, position-aware post-training framework, dubbed textbfRSGround-R1, to progressively enhance spatial understanding.
  arXiv  Detail & Related papers  (2026-01-29T12:35:57Z)
• STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimization [73.98141357780032]
  VLN-CE task requires agents to navigate 3D environments using natural language instructions, without any scene-specific training.<n>Existing methods often fail to achieve robust navigation due to a lack of structured decision-making and insufficient integration of feedback from previous actions.<n>We propose STRIDER, a novel framework that systematically optimize the agent's decision space by integrating spatial layout priors and dynamic task feedback.<n>Our approach introduces two key innovations: 1) a Structured Waypoint Generator that constrains the action space through spatial structure, and 2) a Task-Alignment Regulator that adjusts behavior based on task progress, ensuring semantic alignment throughout navigation.
  arXiv  Detail & Related papers  (2025-10-27T04:37:21Z)
• Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation [64.84996994779443]
  We propose a novel Affordances-Oriented Planner for continuous vision-language navigation (VLN) task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-07-08T12:52:46Z)
• TAS: A Transit-Aware Strategy for Embodied Navigation with Non-Stationary Targets [55.09248760290918]
  We present a novel algorithm for navigation in dynamic scenarios with non-stationary targets.<n>Our novel Transit-Aware Strategy (TAS) enriches embodied navigation policies with object path information.<n> TAS improves performance in non-stationary environments by rewarding agents for synchronizing their routes with target routes.
  arXiv  Detail & Related papers  (2024-03-14T22:33:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.