VL-MemKnG: Hybrid Memory with a Spatio-Temporal Knowledge Graph for Question Answering over Long Egocentric Navigation Trajectories

• URL: http://arxiv.org/abs/2606.17183v1
• Date: Mon, 15 Jun 2026 18:21:14 GMT
• Title: VL-MemKnG: Hybrid Memory with a Spatio-Temporal Knowledge Graph for Question Answering over Long Egocentric Navigation Trajectories
• Abstract summary: We present a hybrid memory framework that extends VL-MemKnG by combining atemporal knowledge graph with persistent segment-level contextual memory.<n>We also introduce WalkieKnowledgeT+, an extension of WalkieKnowledge for long-horizon navigation-oriented video question answering.
• Score: 6.008251172997241
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Answering navigation-relevant questions over long egocentric videos requires retrieving and organizing evidence distributed across distant temporal moments while maintaining spatial and contextual consistency. Although long-context vision--language models can achieve strong answer quality, they are computationally expensive for long trajectories and inefficient for repeated querying. Recent graph-based approaches such as VL-KnG address this challenge through persistent spatio-temporal knowledge graphs, but graph-centric retrieval alone may underrepresent broader temporal continuity and contextual cues. We present VL-MemKnG, a hybrid memory framework that extends VL-KnG by combining a spatio-temporal knowledge graph with persistent segment-level contextual memory. The knowledge graph captures structured relational information and long-range object associations, while segment-level memory preserves broader temporal context for long-horizon evidence retrieval. A hybrid retrieval-and-reasoning module jointly operates over both memory representations to produce evidence-grounded answers and temporally organized supporting evidence. We also introduce WalkieKnowledgeT+, an extension of WalkieKnowledge for long-horizon navigation-oriented video question answering. The benchmark includes temporally distributed reasoning tasks requiring evidence aggregation across multiple non-cooccurring moments. On WalkieKnowledgeT+, VL-MemKnG improves Top-1 retrieval accuracy from 58% to 67% and Recall@1 from 34.50% to 40.55%, outperforming all compared methods, including Gemini 2.5 Pro and Qwen 3.5+. The gains are particularly pronounced on temporal-global and temporally scattered aggregation questions, demonstrating the benefits of combining structured relational memory with segment-level contextual memory while maintaining efficient query-time inference.

Related papers

• Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering [42.31517185994942]
  RAG enables large language models to answer questions by accessing external knowledge.<n>Existing methods suffer from two key challenges: fragmented evidence scattered across temporally distant sessions, and noisy content within retrieved sessions that triggers the lost-in-the-middle effect.<n>We propose MemLoc, a unified Retrieve-Localize-Generate framework for long-term conversational memory QA.
  arXiv  Detail & Related papers  (2026-09-07T06:32:12Z)
• GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience [67.55136435794206]
  A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation.<n>We introduce GROVE, a training-free framework that supports both behaviors with one memory grown causally from a continuous video stream.<n>GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns.
  arXiv  Detail & Related papers  (2026-08-03T15:35:28Z)
• Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios [9.729442664774988]
  Event-Causal RAG is a lightweight retrieval-augmented framework for infinite long-video reasoning.<n>Method segments streaming videos into semantically coherent events and represents each event as a structured State-Event-State graph.<n>On top of this memory, we design a bidirectional retrieval strategy to efficiently identify the most relevant event causal chains.
  arXiv  Detail & Related papers  (2026-05-07T13:01:28Z)
• NaviRAG: Towards Active Knowledge Navigation for Retrieval-Augmented Generation [50.16741209529908]
  NaviRAG is a novel framework that shifts from passive segment retrieval to active knowledge navigation.<n>We show that NaviRAG consistently improves both retrieval recall and end-to-end answer performance over conventional RAG baselines.
  arXiv  Detail & Related papers  (2026-04-14T14:07:01Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning [18.621823772319154]
  We propose an end-to-end reinforcement learning framework that unifies memory operations and question answering within a single policy.<n>UMA maintains a dual memory representation: a compact core summary for global context and a structured Memory Bank that supports explicit CRUD.<n>Across 13 datasets spanning Ledger-QA, Test-Time Learning, and Accurate Retrieval, UMA substantially outperforms long-context and RAG baselines on dynamic reasoning and learning tasks.
  arXiv  Detail & Related papers  (2026-02-13T16:54:23Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agents [3.9396865837159822]
  HiMem is a hierarchical long-term memory framework for long-horizon dialogues.<n>It supports memory construction, retrieval, and dynamic updating during sustained interactions.<n>Results show HiMem consistently outperforms representative baselines in accuracy, consistency, and long-term reasoning.
  arXiv  Detail & Related papers  (2026-01-10T01:26:01Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory [68.97819665784442]
  We introduce LongMemEval, a benchmark designed to evaluate five core long-term memory abilities of chat assistants.<n>LongMemEval presents a significant challenge to existing long-term memory systems.<n>We present a unified framework that breaks down the long-term memory design into three stages: indexing, retrieval, and reading.
  arXiv  Detail & Related papers  (2024-10-14T17:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.