Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents

• URL: http://arxiv.org/abs/2601.19935v1
• Date: Tue, 13 Jan 2026 06:22:32 GMT
• Title: Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents
• Abstract summary: We introduce textscMem2ActBench, a benchmark for evaluating whether agents can proactively leverage long-term memory to execute tool-based actions.<n>A reverse-generation method produces 400 tool-use tasks, with human evaluation confirming 91.3% are strongly memory-dependent.
• Score: 20.357475946040054
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Model (LLM)-based agents are increasingly deployed for complex, tool-based tasks where long-term memory is critical to driving actions. Existing benchmarks, however, primarily test a angent's ability to passively retrieve isolated facts in response to explicit questions. They fail to evaluate the more crucial capability of actively applying memory to execute tasks. To address this gap, we introduce \textsc{Mem2ActBench}, a benchmark for evaluating whether agents can proactively leverage long-term memory to execute tool-based actions by selecting appropriate tools and grounding their parameters. The benchmark simulates persistent assistant usage, where users mention the same topic across long, interrupted interactions and expect previously established preferences and task states to be implicitly applied. We build the dataset with an automated pipeline that merges heterogeneous sources (ToolACE, BFCL, Oasst1), resolves conflicts via consistency modeling, and synthesizes 2,029 sessions with 12 user--assistant--tool turns on average. From these memory chains, a reverse-generation method produces 400 tool-use tasks, with human evaluation confirming 91.3\% are strongly memory-dependent. Experiments on seven memory frameworks show that current systems remain inadequate at actively utilizing memory for parameter grounding, highlighting the need for more effective approaches to evaluate and improve memory application in task execution.

Related papers

• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• Oracle Agent Memory as an Enterprise Memory Substrate for Long-Horizon AI Agents [3.704983597040439]
  Oracle Agent Memory is a database-native memory substrate built on Oracle Database.<n>Report summarizes LongMemEval results, reaching 93.8% accuracy.
  arXiv  Detail & Related papers  (2026-07-14T18:06:53Z)
• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery [58.64329475683699]
  We argue that long-term memory should be evaluated as an auditable post-interaction artifact.<n>We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users.<n>We find that successful assistance and recoverable memory behave as distinct capabilities.
  arXiv  Detail & Related papers  (2026-06-23T13:52:46Z)
• Learning to Retrieve: Dual-Level Long-Term Memory for Text-to-SQL Agents [63.38073530162493]
  Long-term memory helps agents reuse past experiences.<n> Static methods rely on fixed similaritys that do not optimize downstream utility.<n>We propose MERIT, a dynamic multi-horizon memory retrieval framework.
  arXiv  Detail & Related papers  (2026-05-30T05:44:34Z)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective [21.66000179149483]
  EvoMemBench is a unified benchmark organized along two axes: memory scope and memory content.<n>We compare 15 representative memory methods with strong long-context baselines under a standardized protocol.<n>Results show that current memory systems are still far from a general solution.
  arXiv  Detail & Related papers  (2026-05-18T13:54:38Z)
• TA-Mem: Tool-Augmented Autonomous Memory Retrieval for LLM in Long-Term Conversational QA [8.036549927091286]
  Large Language Model (LLM) has exhibited strong reasoning ability in text-based contexts across various domains.<n>The limitation of context window poses challenges for the model on long-range inference tasks.<n>We introduce a tool-augmented autonomous memory retrieval framework (TA-Mem)<n>The TA-Mem is evaluated on the LoCoMo dataset, achieving significant performance improvements over existing baseline approaches.
  arXiv  Detail & Related papers  (2026-03-10T07:27:01Z)
• AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations [61.6579785305668]
  AMemGym is an interactive environment enabling on-policy evaluation and optimization for memory-driven personalization.<n>Our framework provides a scalable, diagnostically rich environment for advancing memory capabilities in conversational agents.
  arXiv  Detail & Related papers  (2026-03-02T15:15:11Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory [89.65731902036669]
  Evo-Memory is a streaming benchmark and framework for evaluating self-evolving memory in large language model (LLM) agents.<n>We evaluate over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets.
  arXiv  Detail & Related papers  (2025-11-25T21:08:07Z)
• MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning [73.27233666920618]
  We propose MemSearcher, an agent workflow that iteratively maintains a compact memory and combines the current turn with it.<n>At each turn, MemSearcher fuses the user's question with the memory to generate reasoning traces, perform search actions, and update memory to retain only information essential for solving the task.<n>We introduce multi-context GRPO, an end-to-end RL framework that jointly optimize reasoning, search strategies, and memory management of MemSearcher Agents.
  arXiv  Detail & Related papers  (2025-11-04T18:27:39Z)
• MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments [6.12783571098263]
  MEMTRACK is a benchmark designed to evaluate long-term memory and state tracking in multi-platform agent environments.<n>Each benchmark instance provides a chronologically platform-interleaved timeline, with noisy, conflicting, cross-referring information.<n>Our benchmark tests memory capabilities such as acquistion, selection and conflict resolution.
  arXiv  Detail & Related papers  (2025-10-01T18:34:03Z)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions [22.190297901876278]
  We identify four core competencies essential for memory agents: accurate retrieval, test-time learning, long-range understanding, and selective forgetting.<n>Existing benchmarks either rely on limited context lengths or are tailored for static, long-context settings like book-based QA.<n>We introduce MemoryAgentBench, a new benchmark specifically designed for memory agents.
  arXiv  Detail & Related papers  (2025-07-07T17:59:54Z)
• FindingDory: A Benchmark to Evaluate Memory in Embodied Agents [49.18498389833308]
  We introduce a new benchmark for long-range embodied tasks in the Habitat simulator.<n>This benchmark evaluates memory-based capabilities across 60 tasks requiring sustained engagement and contextual awareness.
  arXiv  Detail & Related papers  (2025-06-18T17:06:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.