EngramaBench: Evaluating Long-Term Conversational Memory with Structured Graph Retrieval

• URL: http://arxiv.org/abs/2604.21229v1
• Date: Thu, 23 Apr 2026 02:51:42 GMT
• Title: EngramaBench: Evaluating Long-Term Conversational Memory with Structured Graph Retrieval
• Abstract summary: EngramaBench is a benchmark for long-term conversational memory built around five personas, one hundred multi-session conversations, and one hundred fifty queries.<n>We evaluate Engrama, a graph-structured memory system, against GPT-4o full-context prompting and Mem0, an open-source vector-retrieval memory system.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model assistants are increasingly expected to retain and reason over information accumulated across many sessions. We introduce EngramaBench, a benchmark for long-term conversational memory built around five personas, one hundred multi-session conversations, and one hundred fifty queries spanning factual recall, cross-space integration, temporal reasoning, adversarial abstention, and emergent synthesis. We evaluate Engrama, a graph-structured memory system, against GPT-4o full-context prompting and Mem0, an open-source vector-retrieval memory system. All three use the same answering model (GPT-4o), isolating the effect of memory architecture. GPT-4o full-context achieves the highest composite score (0.6186), while Engrama scores 0.5367 globally but is the only system to score higher than full-context prompting on cross-space reasoning (0.6532 vs. 0.6291, n=30). Mem0 is cheapest but substantially weaker (0.4809). Ablations reveal that the components driving Engrama's cross-space advantage trade off against global composite score, exposing a systems-level tension between structured memory specialization and aggregate optimization.

Related papers

• EM^2Mem: Event-Centric Multimodal Memory for Large Language Models [52.3165591032625]
  We propose EM2Mem, an event-centric multimodal memory framework that aligns heterogeneous evidence to event anchors during memory construction.<n>Across three long-video QA benchmarks, EM2Mem improves average accuracy over the strongest memory baseline by 2.0, 2.4, and 3.7 points.
  arXiv  Detail & Related papers  (2026-09-01T01:38:41Z)
• VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction [92.2925193726566]
  VoiceMem is a memory architecture with a parallel informational left brain, an emotional right brain, and streaming memory I/O mechanisms.<n>Experiments and real-world deployment show three advantages.
  arXiv  Detail & Related papers  (2026-08-26T16:50:05Z)
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations [25.703133924514884]
  Large Language Model (LLM) agents increasingly serve as personal assistants and workplace collaborators.<n>Existing memory systems and benchmarks are built around the dyadic, single-user setup.<n>We introduce GroupMemBench, a benchmark that exposes three properties of group memory.
  arXiv  Detail & Related papers  (2026-05-14T07:38:29Z)
• GRAVITY: Architecture-Agnostic Structured Anchoring for Long-Horizon Conversational Memory [38.10762218232756]
  We introduce GRAVITY, a plug-and-play structured memory module.<n>It extracts three knowledge representations from raw conversational utterances.<n>It injects these representations into the host system's prompt as structured anchoring contexts.
  arXiv  Detail & Related papers  (2026-05-03T02:55:46Z)
• GAAMA: Graph Augmented Associative Memory for Agents [0.0]
  GAAMA is a graph-augmented associative memory system that constructs a concept-mediated hierarchical knowledge graph.<n>On the LoCoMo-10 benchmark (1,540 questions across 10 multi-session conversations), GAAMA achieves 78.9% mean reward, outperforming a tuned RAG baseline.
  arXiv  Detail & Related papers  (2026-03-29T23:33:38Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment [54.72137309071243]
  AlpsBench comprises 2,500 long-term interaction sequences curated from WildChat, paired with human-verified structured memories.<n>We define four pivotal tasks - personalized information extraction, updating, retrieval, and utilization - and establish protocols to evaluate the entire lifecycle of memory management.<n>Our benchmarking of frontier LLMs and memory-centric systems reveals that: (i) models struggle to reliably extract latent user traits; (ii) memory updating faces a performance ceiling even in the strongest models; (iii) retrieval accuracy declines sharply in the presence of large distractor pools; and (iv) while explicit memory mechanisms improve recall, they do not
  arXiv  Detail & Related papers  (2026-03-09T11:06:19Z)
• EverMemBench: Benchmarking Long-Term Interactive Memory in Large Language Models [16.865998112859604]
  We introduce EverMemBench, a benchmark featuring multi-party, multi-group conversations spanning over 1 million tokens.<n>EverMemBench evaluates memory systems across three dimensions through 1,000+ QA pairs.
  arXiv  Detail & Related papers  (2026-02-01T16:13:08Z)
• The AI Hippocampus: How Far are We From Human Memory? [77.04745635827278]
  Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers.<n>Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations.<n>Agentic memory introduces persistent, temporally extended memory structures within autonomous agents.
  arXiv  Detail & Related papers  (2026-01-14T03:24:08Z)
• EvolMem: A Cognitive-Driven Benchmark for Multi-Session Dialogue Memory [63.84216832544323]
  EvolMem is a new benchmark for assessing multi-session memory capabilities of large language models (LLMs) and agent systems.<n>To construct the benchmark, we introduce a hybrid data synthesis framework that consists of topic-initiated generation and narrative-inspired transformations.<n>Extensive evaluation reveals that no LLM consistently outperforms others across all memory dimensions.
  arXiv  Detail & Related papers  (2026-01-07T03:14:42Z)
• Evaluating Long-Term Memory for Long-Context Question Answering [100.1267054069757]
  We present a systematic evaluation of memory-augmented methods using LoCoMo, a benchmark of synthetic long-context dialogues annotated for question-answering tasks.<n>Our findings show that memory-augmented approaches reduce token usage by over 90% while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-10-27T18:03:50Z)
• From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents [79.87304940020256]
  Large Language Models (LLMs) have been widely adopted in conversational agents.<n>MemGAS is a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval.<n> Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks.
  arXiv  Detail & Related papers  (2025-05-26T06:13:07Z)
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory [0.5584627289325719]
  Large Language Models (LLMs) have demonstrated remarkable prowess in generating contextually coherent responses.<n>But their fixed context windows pose fundamental challenges for maintaining consistency over prolonged multi-session dialogues.<n>We introduce Mem0, a scalable memory-centric architecture that addresses this issue by dynamically extracting, consolidating, and retrieving salient information from ongoing conversations.
  arXiv  Detail & Related papers  (2025-04-28T01:46:35Z)
• HEMA : A Hippocampus-Inspired Extended Memory Architecture for Long-Context AI Conversations [0.0]
  Large language models (LLMs) struggle with maintaining coherence in extended conversations spanning hundreds of turns.<n>This paper introduces HEMA, a dual-memory system inspired by human cognitive processes.<n>With 10K indexed chunks, Vector Memory achieves P@5 >= 0.80 and R@50 >= 0.74, doubling the area under the precision-recall curve.
  arXiv  Detail & Related papers  (2025-04-23T14:27:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.