Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations

• URL: http://arxiv.org/abs/2602.19320v1
• Date: Sun, 22 Feb 2026 19:50:01 GMT
• Title: Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations
• Abstract summary: Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions.<n>Existing benchmarks are often underscaled, evaluation metrics are misaligned with semantic utility, performance varies significantly across backbone models.<n>This survey presents a structured analysis of agentic memory from both architectural and system perspectives.
• Score: 6.566696478685466
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and personalization beyond fixed context windows. Despite rapid architectural development, the empirical foundations of these systems remain fragile: existing benchmarks are often underscaled, evaluation metrics are misaligned with semantic utility, performance varies significantly across backbone models, and system-level costs are frequently overlooked. This survey presents a structured analysis of agentic memory from both architectural and system perspectives. We first introduce a concise taxonomy of MAG systems based on four memory structures. Then, we analyze key pain points limiting current systems, including benchmark saturation effects, metric validity and judge sensitivity, backbone-dependent accuracy, and the latency and throughput overhead introduced by memory maintenance. By connecting the memory structure to empirical limitations, this survey clarifies why current agentic memory systems often underperform their theoretical promise and outlines directions for more reliable evaluation and scalable system design.

Related papers

• Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents [97.19033877543522]
  We present a controlled harness evaluation of memory substrates for memory-augmented agents.<n>We instrument 26 performance and efficiency metrics under a unified harness.<n>These findings motivate substrate routing as a necessary component of adaptive agent memory systems.
  arXiv  Detail & Related papers  (2026-08-15T03:29:48Z)
• Are We Ready For An Agent-Native Memory System? [37.760278978612874]
  Memory for large language model (LLM) agents has evolved from simple retrieval-augmented mechanisms into a data management system.<n>Existing evaluations benchmark agent memory mainly through end-to-end task success metrics.<n>We present a systematic experimental study of agent memory from a data management perspective.
  arXiv  Detail & Related papers  (2026-06-23T16:34:55Z)
• The Dynamic Gist-Based Memory Model (DGMM): A Memory-Centric Architecture for Artificial Intelligence [0.0]
  We introduce the Dynamic Gist-Based Memory Model (DGMM), an architecture in which experience is represented as an evolving, graph-structured episodic-semantic memory.<n>The results specify properties of DGMM, including episodic persistence, locality of cue-conditioned surprise, and contextual variability without structural modification of stored memory.
  arXiv  Detail & Related papers  (2026-05-04T00:02:51Z)
• MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios [33.8882826707344]
  MemGround is a rigorous long-term memory benchmark grounded in rich, gamified interactive scenarios.<n>To comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite.
  arXiv  Detail & Related papers  (2026-03-23T02:57:39Z)
• SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions [0.0]
  This paper provides the first unified framework for understanding these autonomous systems.<n>We formalize agentic retrieval-generation loops as finite-horizon partially observable Markov decision processes.<n>We identify severe systemic risks inherent to autonomous loops, including compounding hallucination propagation, memory poisoning, retrieval misalignment, and cascading tool-execution vulnerabilities.
  arXiv  Detail & Related papers  (2026-03-07T23:38:57Z)
• RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies [54.23445842621374]
  Memory is critical for long-horizon and history-dependent robotic manipulation.<n>Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms.<n>We introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models.
  arXiv  Detail & Related papers  (2026-03-04T21:59:32Z)
• AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations [61.6579785305668]
  AMemGym is an interactive environment enabling on-policy evaluation and optimization for memory-driven personalization.<n>Our framework provides a scalable, diagnostically rich environment for advancing memory capabilities in conversational agents.
  arXiv  Detail & Related papers  (2026-03-02T15:15:11Z)
• Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey [211.01908189012184]
  Memory, with hundreds of papers released this year, emerges as the critical solution to fill the utility gap.<n>We provide a unified view of foundation agent memory along three dimensions.<n>We then analyze how memory is instantiated and operated under different agent topologies.
  arXiv  Detail & Related papers  (2026-01-14T07:38:38Z)
• The AI Hippocampus: How Far are We From Human Memory? [77.04745635827278]
  Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers.<n>Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations.<n>Agentic memory introduces persistent, temporally extended memory structures within autonomous agents.
  arXiv  Detail & Related papers  (2026-01-14T03:24:08Z)
• From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level [38.24989792739013]
  We present RepoReason, a diagnostic benchmark centered on abductive assertion verification.<n>We implement an execution-driven mutation framework that utilizes the environment as a semantic to regenerate ground-truth states.<n>Our findings provide granular white-box insights for optimizing the next generation of agentic software engineering.
  arXiv  Detail & Related papers  (2026-01-07T09:22:28Z)
• Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory [11.649831360197531]
  We present an experimental, system-oriented analysis of long-term dialog memory architectures.<n>We introduce a unified framework that decomposes dialog memory systems into core components and supports both graph-based and non-graph approaches.
  arXiv  Detail & Related papers  (2026-01-03T20:39:39Z)
• PISA: A Pragmatic Psych-Inspired Unified Memory System for Enhanced AI Agency [50.712873697511206]
  Existing work often lacks adaptability to diverse tasks and overlooks the constructive and task-oriented role of AI agent memory.<n>We propose PISA, a pragmatic, psych-inspired unified memory system that treats memory as a constructive and adaptive process.<n>Our empirical evaluation, conducted on the existing LOCOMO benchmark and our newly proposed AggQA benchmark for data analysis tasks, confirms that PISA sets a new state-of-the-art by significantly enhancing adaptability and long-term knowledge retention.
  arXiv  Detail & Related papers  (2025-10-12T10:34:35Z)
• Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies [0.0]
  This survey aims to categorizing visual reasoning into five major types (relational, symbolic, temporal, causal, and commonsense)<n>We review evaluation protocols designed to assess functional correctness, structural consistency, and causal validity, and critically analyze their limitations in terms of generalizability, explanatory power.
  arXiv  Detail & Related papers  (2025-08-14T10:53:35Z)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions [22.190297901876278]
  We identify four core competencies essential for memory agents: accurate retrieval, test-time learning, long-range understanding, and selective forgetting.<n>Existing benchmarks either rely on limited context lengths or are tailored for static, long-context settings like book-based QA.<n>We introduce MemoryAgentBench, a new benchmark specifically designed for memory agents.
  arXiv  Detail & Related papers  (2025-07-07T17:59:54Z)
• MemOS: A Memory OS for AI System [116.87568350346537]
  Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI)<n>Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.<n>MemOS is a memory operating system that treats memory as a manageable system resource.
  arXiv  Detail & Related papers  (2025-07-04T17:21:46Z)
• Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic [25.070876549371693]
  We present a logic that enables reasoning over perception data using spatial and temporal operators. One major advantage ofSTPL is that it facilitates basic sanity checks on the functional performance of the perception system.
  arXiv  Detail & Related papers  (2022-06-29T02:36:53Z)
• Exploring and Analyzing Machine Commonsense Benchmarks [0.13999481573773073]
  We argue that the lack of a common vocabulary for aligning these approaches' metadata limits researchers in their efforts to understand systems' deficiencies. We describe our initial MCS Benchmark Ontology, an common vocabulary that formalizes benchmark metadata.
  arXiv  Detail & Related papers  (2020-12-21T19:01:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.