INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval

• URL: http://arxiv.org/abs/2602.12971v1
• Date: Fri, 13 Feb 2026 14:45:55 GMT
• Title: INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval
• Abstract summary: INHerit-SG is a semantic scene graph that supports interpretable human-intent reasoning in complex environments.<n>An event-triggered map update mechanism reorganizes the graph only when meaningful semantic events occur.<n>This explicit interpretability improves the success rate and reliability of complex retrievals, enabling the system to adapt to a broader spectrum of human interaction tasks.
• Score: 32.38739911399099
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Driven by advancements in foundation models, semantic scene graphs have emerged as a prominent paradigm for high-level 3D environmental abstraction in robot navigation. However, existing approaches are fundamentally misaligned with the needs of embodied tasks. As they rely on either offline batch processing or implicit feature embeddings, the maps can hardly support interpretable human-intent reasoning in complex environments. To address these limitations, we present INHerit-SG. We redefine the map as a structured, RAG-ready knowledge base where natural-language descriptions are introduced as explicit semantic anchors to better align with human intent. An asynchronous dual-process architecture, together with a Floor-Room-Area-Object hierarchy, decouples geometric segmentation from time-consuming semantic reasoning. An event-triggered map update mechanism reorganizes the graph only when meaningful semantic events occur. This strategy enables our graph to maintain long-term consistency with relatively low computational overhead. For retrieval, we deploy multi-role Large Language Models (LLMs) to decompose queries into atomic constraints and handle logical negations, and employ a hard-to-soft filtering strategy to ensure robust reasoning. This explicit interpretability improves the success rate and reliability of complex retrievals, enabling the system to adapt to a broader spectrum of human interaction tasks. We evaluate INHerit-SG on a newly constructed dataset, HM3DSem-SQR, and in real-world environments. Experiments demonstrate that our system achieves state-of-the-art performance on complex queries, and reveal its scalability for downstream navigation tasks. Project Page: https://fangyuktung.github.io/INHeritSG.github.io/

Related papers

• Query-Aware Flow Diffusion for Graph-Based RAG with Retrieval Guarantees [10.806910118253976]
  Graph-based Retrieval-Augmented Generation (RAG) systems leverage interconnected knowledge structures to capture complex relationships.<n>We propose Query-Aware Flow Diffusion RAG (QAFD-RAG), a training-free framework that dynamically adapts graph traversal to each query's holistic semantics.<n>QAFD-RAG recovers relevant subgraphs with high probability under mild signal-to-noise conditions.
  arXiv  Detail & Related papers  (2026-04-21T08:53:34Z)
• MDER-DR: Multi-Hop Question Answering with Entity-Centric Summaries [2.756584457554517]
  Map-Disambiguate-Enrich-Reduce (MDER) generates context-derived triple descriptions and integrates them with entity-level summaries.<n>Decompose-Resolve (DR) decomposes user queries into resolvable triples and grounds them in the Knowledge Graph.<n>MDER and DR form an LLM-driven QA pipeline that is robust to sparse, incomplete, and complex relational data.
  arXiv  Detail & Related papers  (2026-03-11T18:38:44Z)
• OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models [57.94189874119267]
  Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems.<n>Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm.<n>We propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language.
  arXiv  Detail & Related papers  (2026-01-19T12:23:44Z)
• Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Models [99.85131798240808]
  We introduce a novel generative framework called textitGuided Topology Diffusion (GTD)<n>Inspired by conditional discrete graph diffusion models, GTD formulates topology synthesis as an iterative construction process.<n>At each step, the generation is steered by a lightweight proxy model that predicts multi-objective rewards.<n>Experiments show that GTD can generate highly task-adaptive, sparse, and efficient communication topologies.
  arXiv  Detail & Related papers  (2025-10-09T05:28:28Z)
• GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion [33.66085762717581]
  LLMs excel in localized code completion but struggle with repository-level tasks due to limited context windows.<n> GRACE builds a multi-level, multi-semantic code graph to capture both static and dynamic code semantics.<n>Experiments demonstrate that GRACE significantly outperforms state-of-the-art methods across all metrics.
  arXiv  Detail & Related papers  (2025-09-07T09:01:48Z)
• Assemble Your Crew: Automatic Multi-agent Communication Topology Design via Autoregressive Graph Generation [91.17994756436259]
  Multi-agent systems (MAS) based on large language models (LLMs) have emerged as a powerful solution for dealing with complex problems across diverse domains.<n>Existing approaches are fundamentally constrained by their reliance on a template graph modification paradigm with a predefined set of agents and hard-coded interaction structures.<n>We propose ARG-Designer, a novel autoregressive model that operationalizes this paradigm by constructing the collaboration graph from scratch.
  arXiv  Detail & Related papers  (2025-07-24T09:17:41Z)
• SlimRAG: Retrieval without Graphs via Entity-Aware Context Selection [38.200971604630524]
  SlimRAG is a lightweight framework for retrieval without graphs.<n>It replaces structure-heavy components with a simple yet effective entity-aware mechanism.<n> Experiments show that SlimRAG outperforms strong flat and graph-based baselines in accuracy.
  arXiv  Detail & Related papers  (2025-06-15T15:36:17Z)
• Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generation [69.45495166424642]
  We develop a robust and discriminative QA benchmark to measure temporal, causal, and character consistency understanding in narrative documents.<n>We then introduce Entity-Event RAG (E2RAG), a dual-graph framework that keeps separate entity and event subgraphs linked by a bipartite mapping.<n>Across ChronoQA, our approach outperforms state-of-the-art unstructured and KG-based RAG baselines, with notable gains on causal and character consistency queries.
  arXiv  Detail & Related papers  (2025-06-06T10:07:21Z)
• Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning [62.640169289390535]
  SPLIT-RAG is a multi-agent RAG framework that addresses the limitations with question-driven semantic graph partitioning and collaborative subgraph retrieval.<n>The innovative framework first create Semantic Partitioning of Linked Information, then use the Type-Specialized knowledge base to achieve Multi-Agent RAG.<n>The attribute-aware graph segmentation manages to divide knowledge graphs into semantically coherent subgraphs, ensuring subgraphs align with different query types.<n>A hierarchical merging module resolves inconsistencies across subgraph-derived answers through logical verifications.
  arXiv  Detail & Related papers  (2025-05-20T06:44:34Z)
• SG-Reg: Generalizable and Efficient Scene Graph Registration [23.3853919684438]
  We design a scene graph network to encode multiple modalities of semantic nodes.<n>In the back-end, we employ a robust pose estimator to decide transformation according to the correspondences.<n>Our method achieves a slightly higher registration recall while requiring only 52 KB of communication bandwidth for each query frame.
  arXiv  Detail & Related papers  (2025-04-20T01:22:40Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.