MemConflict: Evaluating Long-Term Memory Systems Under Memory Conflicts

• URL: http://arxiv.org/abs/2605.20926v1
• Date: Wed, 20 May 2026 09:11:04 GMT
• Title: MemConflict: Evaluating Long-Term Memory Systems Under Memory Conflicts
• Abstract summary: We propose a diagnostic framework that treats memory validity as a query-conditioned fitness-for-use problem.<n>MemConflict formalizes dynamic, static, and conditional conflicts over temporal validity, factual correctness, and contextual applicability.<n>It simulates controlled long-horizon histories from structured user profiles, introduces cross-session conflicts, and injects semantically similar distractors to create competition among memory candidates.
• Score: 19.9199366981741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-term memory systems enable conversational agents based on large language models (LLMs) to retain, retrieve, and apply user-specific information across multi-session interactions. However, existing evaluations mainly assess outcome-level performance or temporal updating, providing limited insight into how systems retrieve and rank temporally valid, factually correct, and contextually applicable memory evidence under conflicting alternatives. To address this gap, we propose MemConflict, a diagnostic framework that treats memory validity as a query-conditioned fitness-for-use problem. MemConflict formalizes dynamic, static, and conditional conflicts over temporal validity, factual correctness, and contextual applicability. It simulates controlled long-horizon histories from structured user profiles, introduces cross-session conflicts, and injects semantically similar distractors to create competition among memory candidates. The resulting multi-session dialogue benchmark supports black-box evaluation of final answers and white-box analysis of supporting-memory retrieval and ranking. Experiments on six representative long-term memory systems show uneven strengths across conflict types, with answer correctness often diverging from memory retrieval and ranking. Sensitivity analyses reveal that longer histories, distractors, implicit queries, and larger conflict distances degrade performance. Diagnostics show failures from missing supporting memories and ineffective use of retrieved memories. Collectively, MemConflict advances principled long-term memory governance through retrieval-aware, conflict-aware reliability assessment.

Related papers

• UTILMEM: Benchmarking Evidence Utilization in Long-Term Conversational Memory [32.625416782173204]
  We introduce UtilMem, a diagnostic benchmark to evaluate memory utilization.<n>We find that strong performance on conventional factual-memory benchmarks does not reliably translate into effective memory utilization.<n>These findings expose a substantial gap between accessing stored information and using it effectively, and suggest that progress in long-term conversational memory will require architectures that explicitly support evidence integration and robustness to retrieval interference.
  arXiv  Detail & Related papers  (2026-08-31T09:41:26Z)
• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models [56.31411457917676]
  We introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.<n>Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods.
  arXiv  Detail & Related papers  (2026-05-27T06:04:19Z)
• MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems [69.06764269022925]
  We study how current memory-augmented agents perform in realistic, interference-heavy, long-horizon settings.<n> MINTEval is a benchmark featuring long, highly interconnected contexts with frequently updated information.<n>Overall, MINTEval has 15.6k question-answering pairs over long-horizon contexts averaging 138.8k tokens and extending to 1.8M tokens per instance.
  arXiv  Detail & Related papers  (2026-05-18T15:43:35Z)
• Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents [0.33842793760651557]
  We propose a causal memory-selection technique that estimates how candidate memories affect the model's answer under controlled interventions.<n>Results show that CMI achieves a stronger balance between answer quality and robustness to misleading memory, suggesting that reliable long-term memory requires selecting context based on causal usefulness rather than relevance alone.
  arXiv  Detail & Related papers  (2026-05-17T20:21:55Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• MemWeaver: Weaving Hybrid Memories for Traceable Long-Horizon Agentic Reasoning [26.119505362626338]
  We propose a unified memory framework that consolidates long-term agent experiences into three interconnected components.<n>MemWeaver employs a dual-channel retrieval strategy that jointly retrieves structured knowledge and supporting evidence to construct compact yet information-dense contexts.
  arXiv  Detail & Related papers  (2026-01-26T06:39:27Z)
• Amory: Building Coherent Narrative-Driven Agent Memory through Agentic Reasoning [14.368376032599437]
  Amory is a working memory framework that actively constructs structured memory representations during offline time.<n>Amory organizes conversational fragments into episodic narratives, consolidates memories with momentum, and semanticizes peripheral facts into semantic memory.<n>Amory achieves considerable improvements over previous state-of-the-art, with performance comparable to full context reasoning while reducing response time by 50%.
  arXiv  Detail & Related papers  (2026-01-09T19:51:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.