Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

• URL: http://arxiv.org/abs/2607.21571v1
• Date: Thu, 23 Jul 2026 17:50:45 GMT
• Title: Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering
• Abstract summary: Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes.<n>We investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries.<n>We find that simply preserving existing memory is often insufficient.<n>We highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation.
• Score: 93.80043825360514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. However, real-world robots operate continuously and must accumulate, retain, and selectively reuse information acquired from prior interactions. Despite this practical requirement, the architectural mechanisms needed to support sequential memory in EQA remain underexplored. In this work, we investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries. We find that simply preserving existing memory is often insufficient. Agents that retain only traversability information, such as 2D occupancy maps, remember where the robot has explored but not the visual-semantic evidence needed for later questions. Agents trained on short-horizon episodic data face a different challenge: when exposed to continuous, multi-query histories, their inherited context suffers from severe temporal mismatch, rather than forming a reusable scene representation. To overcome this architectural bottleneck, we highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation. Extensive experiments in simulated environments reveal that this form of memory breaks the accuracy-efficiency tradeoff in sequential settings, simultaneously achieving higher answer accuracy and lower navigation costs. We further validate these findings on a real-world mobile robot, demonstrating that spatially grounded visual memory is critical for enabling continuous, intelligent operation in physical environments.

Related papers

• HitMem: Hierarchical Temporal 3D Memory with Multi-Modal Context-Aware Retrieval for Dynamic Environments [5.55387168911911]
  HitMem is a hierarchical temporal 3D memory framework with a multi-modal context-aware retrieval mechanism.<n>We show that HitMem significantly improves object relocation accuracy, reduces exploration costs, and enhances task execution performance in dynamic environments.
  arXiv  Detail & Related papers  (2026-09-01T09:09:21Z)
• SuperMemory-VQA: An Egocentric Visual Question-Answering Benchmark for Long-Horizon Memory [14.96864764261795]
  SuperMemory-VQA is a dataset for evaluating AI assistants on practical, long-horizon memory tasks.<n>It contains 52.9 hours of everyday activities recorded with AI glasses, including synchronized RGB video, audio transcription, eye gaze, IMU, and SLAM trajectories.<n>We construct grounded 4,853 question-answer pairs that span object and location memory, intent recall, visual scene recall, timeline reconstruction, conversational memory, and in-context retrieval.
  arXiv  Detail & Related papers  (2026-05-30T17:53:10Z)
• WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction [72.1620416874118]
  Multimodal large language models are increasingly deployed as long-horizon agents.<n>Existing benchmarks measure recall over static dialogue, collapse memory into a single end-of-task accuracy, and reduce visual observations to captions.<n>We formulate multimodal agent memory as an Action-World Interaction Loop with an observable four-stage lifecycle.
  arXiv  Detail & Related papers  (2026-05-28T04:27:20Z)
• MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems [69.06764269022925]
  We study how current memory-augmented agents perform in realistic, interference-heavy, long-horizon settings.<n> MINTEval is a benchmark featuring long, highly interconnected contexts with frequently updated information.<n>Overall, MINTEval has 15.6k question-answering pairs over long-horizon contexts averaging 138.8k tokens and extending to 1.8M tokens per instance.
  arXiv  Detail & Related papers  (2026-05-18T15:43:35Z)
• MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios [33.8882826707344]
  MemGround is a rigorous long-term memory benchmark grounded in rich, gamified interactive scenarios.<n>To comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite.
  arXiv  Detail & Related papers  (2026-03-23T02:57:39Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers [0.42061757959666934]
  Large language model (LLM) agents increasingly operate in settings where a single context window is too small to capture what has happened.<n>Memory is what turns a stateless text generator into a genuinely adaptive agent.<n>This survey offers a structured account of how memory is designed, implemented, and evaluated in modern LLM-based agents.
  arXiv  Detail & Related papers  (2026-03-08T15:08:01Z)
• RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies [54.23445842621374]
  Memory is critical for long-horizon and history-dependent robotic manipulation.<n>Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms.<n>We introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models.
  arXiv  Detail & Related papers  (2026-03-04T21:59:32Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answering [27.218583831470212]
  Long-term Active Question Answering (LA-EQA) is a new task in which a robot must both recall past experiences and actively explore its environment to answer complex, temporally-grounded questions.<n>Standard EQA approaches based on large models struggle in this setting due to limited context windows, absence of persistent memory, and an inability to combine memory recall with active exploration.<n>We propose a structured memory system for robots, inspired by the mind palace method from cognitive science.
  arXiv  Detail & Related papers  (2025-07-17T07:11:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.