PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments

• URL: http://arxiv.org/abs/2603.23231v1
• Date: Tue, 24 Mar 2026 14:04:11 GMT
• Title: PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments
• Abstract summary: We introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall.<n>PerMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time.<n> Experiments demonstrate that by linking related interactions, advanced memory systems can extract more precise preferences and reduce token consumption.
• Score: 72.02445514666428
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. However, prior evaluations typically interleave preference-related dialogues with irrelevant conversations, reducing the task to needle-in-a-haystack retrieval while ignoring relationships between events that drive the evolution of user preferences. Such settings overlook a fundamental characteristic of real-world personalization: preferences emerge gradually and accumulate across interactions within noisy contexts. To bridge this gap, we introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall. Additionally, we incorporate (1) text variability and (2) linguistic alignment to simulate erratic user inputs and individual idiolects in real-world data. PERMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time. We design both multiple-choice and interactive tasks to probe the model's understanding of persona along the interaction timeline. Experiments demonstrate that by linking related interactions, advanced memory systems can extract more precise preferences and reduce token consumption, outperforming traditional semantic retrieval of raw dialogues. Nevertheless, they still struggle to maintain a coherent persona across temporal depth and cross-domain interference, highlighting the need for more robust personalized memory management in agents. Our code and data are open-sourced at https://github.com/PolarisLiu1/PERMA.

Related papers

• QUMem: Personalized Memory for Query-Conditioned User-State Inference in LLM Agents [24.465097221083777]
  textscQUMem is a structured memory framework for query-conditioned user-state inference.<n>textscQUMem achieves state-of-the-art performance on both PersonaMem and KnowU-Bench.
  arXiv  Detail & Related papers  (2026-08-17T06:34:24Z)
• Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory [54.947805187562274]
  We introduce realistic dialogues across diverse interaction scenarios that exhibit dynamic temporal evolution and long-term coherence.<n>The resulting benchmark encompasses challenging question-answer pairs spanning seven inquiry types.<n>We identify at least one of 27 critical memory characteristics that we identify as essential yet underexplored in current research.
  arXiv  Detail & Related papers  (2026-05-29T09:51:22Z)
• VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions [63.13827503828231]
  We introduce VitaBench 2.0, a benchmark for evaluating personalized and proactive agent behavior in long-term user interactions.<n>Results show that real-world personalization remains highly challenging even for state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-26T15:07:38Z)
• Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactions [17.9008221917999]
  POLAR is a memory-augmented framework for personalized embodied agents over long-term user interactions.<n>To execute embodied tasks, POLAR retrieves relevant memories to interpret the current request and guide task execution.<n>Results show that the proposed memory mechanism consistently improves performance by enabling more effective use of information accumulated over prior interactions.
  arXiv  Detail & Related papers  (2026-05-25T18:27:27Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• Learning Personalized Agents from Human Feedback [36.47803872623135]
  We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization.<n>PAHF learns online from live interaction using explicit per-user memory.<n> benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts.
  arXiv  Detail & Related papers  (2026-02-18T04:18:47Z)
• OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agents [55.27061195244624]
  We formalize over-personalization into three types: Irrelevance, Repetition, and Sycophancy.<n>Agents tend to retrieve and over-attend to user memories even when unnecessary.<n>Our work takes an initial step toward more controllable and appropriate personalization in memory-augmented dialogue systems.
  arXiv  Detail & Related papers  (2026-01-20T08:27:13Z)
• Towards Proactive Personalization through Profile Customization for Individual Users in Dialogues [28.522406727886395]
  PersonalAgent is a lifelong agent designed to continuously infer and adapt to user preferences.<n>Experiments show that PersonalAgent achieves superior performance over strong prompt-based and policy optimization baselines.<n>Our findings underscore the importance of lifelong personalization for developing more inclusive and adaptive conversational agents.
  arXiv  Detail & Related papers  (2025-12-17T10:47:06Z)
• Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction [55.24448139349266]
  We present PAL-Bench, a new benchmark designed to evaluate the personalization capabilities of service-oriented assistants in long-term user-agent interactions.<n>To improve personalized service-oriented interactions, we propose H$2$Memory, a hierarchical and heterogeneous memory framework.
  arXiv  Detail & Related papers  (2025-11-17T14:22:32Z)
• RGMem: Renormalization Group-based Memory Evolution for Language Agent User Profile [8.224917568034572]
  We propose a self-evolving memory framework, inspired by the ideology of classic renormalization group (RG) in physics.<n>This framework enables to organize the dialogue history in multiple scales.<n>The core innovation of our work lies in modeling memory evolution as a multi-scale process of information compression and emergence.
  arXiv  Detail & Related papers  (2025-10-18T08:16:46Z)
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots [62.295373408415365]
  We propose a personalized hybrid matching network (PHMN) for context-response matching. Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information. We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
  arXiv  Detail & Related papers  (2021-03-17T09:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.