MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

• URL: http://arxiv.org/abs/2605.15128v1
• Date: Thu, 14 May 2026 17:37:52 GMT
• Title: MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
• Abstract summary: Existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning.<n>We introduce MemEye, a framework that evaluates memory capabilities from two dimensions.<n>Under this framework, we construct a new benchmark across 8 life-scenario tasks.
• Score: 80.97855900579512
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-term agent memory is increasingly multimodal, yet existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning. In prior work, many visually grounded questions can be answered using only captions or textual traces, allowing answers to be inferred without preserving the fine-grained visual evidence. Meanwhile, harder cases that require reasoning over changing visual states are largely absent. Therefore, we introduce MemEye, a framework that evaluates memory capabilities from two dimensions: one measures the granularity of decisive visual evidence (from scene-level to pixel-level evidence), and the other measures how retrieved evidence must be used (from single evidence to evolutionary synthesis). Under this framework, we construct a new benchmark across 8 life-scenario tasks, with ablation-driven validation gates for assessing answerability, shortcut resistance, visual necessity, and reasoning structure. By evaluating 13 memory methods across 4 VLM backbones, we show that current architectures still struggle to preserve fine-grained visual details and reason about state changes over time. Our findings show that long-term multimodal memory depends on evidence routing, temporal tracking, and detail extraction.

Related papers

• MedCache: Efficient and Temporally Valid Memory for Longitudinal Clinical Agents [21.198155448403444]
  We introduce a benchmark of multi-visit, multi-context patient records that evaluates long-context evidence retrieval, cross-time evidence aggregation, and cross-context clinical reasoning.<n>Using this benchmark, we systematically study four memory design choices: curation, organization, retrieval, and memory-augmented reasoning.<n>We propose textitMedCache, a hybrid framework that constructs temporally valid patient memory, organizes evidence into overlapping specialty views, routes each query to relevant memories, and adaptively invokes one or multiple specialists.
  arXiv  Detail & Related papers  (2026-08-30T03:28:56Z)
• REVEAL: A Rubric-Guided Agent for Explicit Evidence Sufficiency Verificationin Long-Video Question Answering [25.519927761448603]
  REVEAL is a rubric-guided agent framework for long-video question answering.<n>We introduce an adaptive visual-similarity-based preprocessing pipeline that groups visually coherent adjacent frames into natural event units.<n>Built upon this structured memory, REVEAL uses an automatically constructed rubric library to explicitly verify whether retrieved evidence satisfies sufficiency criteria.
  arXiv  Detail & Related papers  (2026-08-09T09:55:42Z)
• ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding [11.461394531496397]
  ViSAGE is a multimodal agentic memory framework that constructs self-correcting, entity-centric memories.<n>It applies bidirectional memory refinement to propagate delayed identity evidence, retroactively unifying historical records and improving future reasoning.<n>Extensive results demonstrate that ViSAGE consistently outperforms the strongest baseline, achieving 5.9% higher accuracy.
  arXiv  Detail & Related papers  (2026-07-29T10:25:23Z)
• MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents [26.87993979719045]
  Memory-augmented LLM agents typically answer queries by retrieving relevant memories and feeding them directly to an answer model.<n>We propose MemChain, a trainable post-retrieval memory policy that transforms retrieved candidates into answer-facing active memory.<n>We show that MemChain consistently achieves state-of-the-art performance across both closed-source and open-weight frozen answer models.
  arXiv  Detail & Related papers  (2026-07-27T07:37:43Z)
• Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering [93.80043825360514]
  Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes.<n>We investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries.<n>We find that simply preserving existing memory is often insufficient.<n>We highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation.
  arXiv  Detail & Related papers  (2026-07-23T17:50:45Z)
• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction [72.1620416874118]
  Multimodal large language models are increasingly deployed as long-horizon agents.<n>Existing benchmarks measure recall over static dialogue, collapse memory into a single end-of-task accuracy, and reduce visual observations to captions.<n>We formulate multimodal agent memory as an Action-World Interaction Loop with an observable four-stage lifecycle.
  arXiv  Detail & Related papers  (2026-05-28T04:27:20Z)
• MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models [50.25006399944962]
  Memory is essential for large vision-language models to handle long, multimodal interactions.<n>MEMLENS is a benchmark for memory in multimodal multi-session conversations.<n>We evaluate 27 LVLMs and 7 memory-augmented agents.
  arXiv  Detail & Related papers  (2026-05-14T14:41:17Z)
• EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding [89.26501160264199]
  EgoMemReason systematically evaluates week-long egocentric video understanding through memory-driven reasoning.<n>EgoMemReason comprises 500 questions across three memory types and six core challenges.<n>We evaluate EgoMemReason on 17 methods across MLLMs and agentic frameworks.
  arXiv  Detail & Related papers  (2026-05-11T01:59:59Z)
• MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios [33.8882826707344]
  MemGround is a rigorous long-term memory benchmark grounded in rich, gamified interactive scenarios.<n>To comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite.
  arXiv  Detail & Related papers  (2026-03-23T02:57:39Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• MemWeaver: Weaving Hybrid Memories for Traceable Long-Horizon Agentic Reasoning [26.119505362626338]
  We propose a unified memory framework that consolidates long-term agent experiences into three interconnected components.<n>MemWeaver employs a dual-channel retrieval strategy that jointly retrieves structured knowledge and supporting evidence to construct compact yet information-dense contexts.
  arXiv  Detail & Related papers  (2026-01-26T06:39:27Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• Agentic Learner with Grow-and-Refine Multimodal Semantic Memory [50.81667005063605]
  ViLoMem is a dual-stream memory framework that constructs compact, schema-based memory.<n>It encodes visual distraction patterns and logical reasoning errors, enabling MLLMs to learn from their successful and failed experiences.
  arXiv  Detail & Related papers  (2025-11-26T18:55:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.