StratMem-Bench: Evaluating Strategic Memory Use in Virtual Character Conversation Beyond Factual Recall

• URL: http://arxiv.org/abs/2604.26243v1
• Date: Wed, 29 Apr 2026 02:55:20 GMT
• Title: StratMem-Bench: Evaluating Strategic Memory Use in Virtual Character Conversation Beyond Factual Recall
• Abstract summary: StratMem-Bench is a new benchmark to evaluate strategic memory use in character-centric dialogues.<n>This dataset comprises 657 instances where virtual characters must navigate heterogeneous memory pools.
• Score: 11.978501031062159
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Achieving realistic human-like conversation for virtual characters requires not only a simple memorization and recall of past events, but also the strategic utilization of memory to meet factual needs and social engagement. Current memory utilization relevant (e.g., memory-augmented generation, long-term dialogue, and etc.) benchmarks overlook this nuance, treating memory primarily as a static repository of facts rather than a dynamic resource to be strategically deployed in dialogues. To address this gap, we design StratMem-Bench, a new benchmark to evaluate strategic memory use in character-centric dialogues. This dataset comprises 657 instances where virtual characters must navigate heterogeneous memory pools containing required, supportive, and irrelevant memories. We also propose a framework with different evaluation metrics including Strict Memory Compliance, Memory Integration Quality, Proactive Enrichment Score and Conditional Irrelevance Rate, to evaluate strategic memory use capabilities of virtual characters. Experiments on StratMem-Bench which leverage the state-of-the-art large language models as virtual characters show that all models perform well at distinguishing between required and irrelevant memories, but struggle once supportive memories are introduced into the decision process.

Related papers

• Mechanistic Attention Guidance for Agent Memory Refinement [32.57216046158823]
  We show that retrieval-head attention provides a mechanistic signal for revealing segment-level memory utilization.<n>We propose Attention-Guided Memory Refinement (AGMR), a framework that uses utilization patterns revealed by attention to guide targeted segment-level memory updates.
  arXiv  Detail & Related papers  (2026-07-20T07:17:50Z)
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models [56.31411457917676]
  We introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.<n>Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods.
  arXiv  Detail & Related papers  (2026-05-27T06:04:19Z)
• Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents [20.22872890297194]
  Existing large language model (LLM) based memory systems apply universal, static policies that overlook a fundamental reality: the contexts that are worth storing in memory are different across users.<n>We introduce PerMemBench, the first benchmark for evaluating personalized memory systems.<n>We propose session level storage gating, a lightweight framework that selectively bypasses memory operations for transient sessions.
  arXiv  Detail & Related papers  (2026-05-25T07:48:33Z)
• LMEB: Long-horizon Memory Embedding Benchmark [49.57481835614834]
  We introduce the Long-horizon Memory Embedding Benchmark (LMEB), a comprehensive framework that evaluates embedding models' capabilities.<n>LMEB spans 22 datasets and 193 zero-shot retrieval tasks across 4 memory types.<n>We evaluate 15 widely used embedding models, ranging from hundreds of millions to ten billion parameters.
  arXiv  Detail & Related papers  (2026-03-13T02:09:57Z)
• RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design [77.30163153176954]
  RMBench is a simulation benchmark comprising 9 manipulation tasks that span multiple levels of memory complexity.<n>Mem-0 is a modular manipulation policy with explicit memory components designed to support controlled ablation studies.<n>We identify memory-related limitations in existing policies and provide empirical insights into how architectural design choices influence memory performance.
  arXiv  Detail & Related papers  (2026-03-01T18:59:59Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• Graph-based Agent Memory: Taxonomy, Techniques, and Applications [63.70340159016138]
  Memory emerges as the core module in the Large Language Model (LLM)-based agents for long-horizon complex tasks.<n>Among diverse paradigms, graph stands out as a powerful structure for agent memory due to the intrinsic capabilities to model relational dependencies.<n>This survey presents a comprehensive review of agent memory from the graph-based perspective.
  arXiv  Detail & Related papers  (2026-02-05T13:49:05Z)
• Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity [26.512226057571947]
  Memora is a harmonic memory representation that structurally balances abstraction and specificity.<n>We show that Memora establishes a new state-of-the-art on the LoCoMo and LongMemEval benchmarks, demonstrating better retrieval relevance and reasoning effectiveness as memory scales.
  arXiv  Detail & Related papers  (2026-02-03T09:44:43Z)
• Evaluating Long-Term Memory for Long-Context Question Answering [100.1267054069757]
  We present a systematic evaluation of memory-augmented methods using LoCoMo, a benchmark of synthetic long-context dialogues annotated for question-answering tasks.<n>Our findings show that memory-augmented approaches reduce token usage by over 90% while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-10-27T18:03:50Z)
• Multiple Memory Systems for Enhancing the Long-term Memory of Agent [9.43633399280987]
  Existing methods, such as MemoryBank and A-MEM, have poor quality of stored memory content.<n>We have designed a multiple memory system inspired by cognitive psychology theory.
  arXiv  Detail & Related papers  (2025-08-21T06:29:42Z)
• MemoCue: Empowering LLM-Based Agents for Human Memory Recall via Strategy-Guided Querying [12.524353050278105]
  We propose a strategy-guided agent-assisted memory recall method, allowing the agent to transform an original query into a cue-rich one.<n>We design a 5W Recall Map to classify memory queries into five typical scenarios.<n>We then propose a hierarchical recall tree combined with the Monte Carlo Tree Search algorithm to optimize the selection of strategy and the generation of strategy responses.
  arXiv  Detail & Related papers  (2025-07-31T15:11:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.