FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents

• URL: http://arxiv.org/abs/2608.04095v1
• Date: Tue, 04 Aug 2026 18:00:04 GMT
• Title: FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents
• Abstract summary: FinPerMA is an event-grounded benchmark that evaluates personalized memory against frozen longitudinal investor trajectories.<n>No full-context configuration exceeds approximately 0.47 overall accuracy or approximately 39% on multiple-choice questions.<n> Attribution analysis shows that summary-based memory often preserves factual details while losing the preference signals needed for personalization.
• Score: 9.305912044747368
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents are increasingly used as personalized assistants in high-stakes domains such as financial advising, yet it remains unclear whether they can maintain and update an individualized user model over long horizons. Existing personalized-memory benchmarks primarily test factual retention or rely on weakly constrained model-generated trajectories, leaving event-driven preference adaptation underexplored. We introduce FinPerMA, an event-grounded benchmark that evaluates personalized memory against frozen longitudinal investor trajectories. Its generation pipeline combines deterministic, theory-informed impact rules, controlled LLM narration, and automated quality screening; a Post-Shock checkpoint isolates whether an agent has integrated a material event into its persistent user model. On 2,994 questions from 276 personas, seven frontier LLMs and up to seven memory configurations remain far from saturated: no full-context configuration exceeds approximately 0.47 overall accuracy or approximately 39% on multiple-choice questions. Attribution analysis shows that summary-based memory often preserves factual details while losing the preference signals needed for personalization; simple retrieval can therefore outperform purpose-built memory systems, with the gap widening after shocks.

Related papers

• Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving [0.8419406375517101]
  VLMs have shown promise for autonomous driving yet still suffer from weak-temporal perception, and limited generalization.<n>Recent methods improve reasoning and decision-making through CoT explanations, retrieval-augmented generation or the static injection of tool outputs.<n>We present to our knowledge, the first synergistic framework that tightly couples hierarchical memory with proactive tool invocation in a closed loop.
  arXiv  Detail & Related papers  (2026-09-08T03:59:10Z)
• Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner [55.80248281761638]
  multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning.<n>We propose DG-Mem, a dual-grained agentic memory framework that augments a frozen MLLM with a non- externally stored memory built once from training-time rollouts and consulted read-only at test time.
  arXiv  Detail & Related papers  (2026-08-24T13:56:01Z)
• Long-Horizon Embodied Decision-Making via Multimodal Memory Compression [71.16335993005139]
  DunphyBench is a new benchmark for evaluating agents on long-horizon human-centered embodied decision-making.<n>MeMento is a preference-conditioned multimodal memory compressor that selectively compresses decision-relevant information based on user preferences.<n>MeMento helps VLM-driven agents improve accuracy by 7.18%, while reducing memory usage by 85.38% compared to the strongest baseline.
  arXiv  Detail & Related papers  (2026-08-02T19:29:48Z)
• Multi-Head Recurrent Memory Agents [6.282017902182834]
  Multi-Head Recurrent Memory partitions memory into independent heads governed by a stage-wise select-then-update strategy.<n>As a lightweight instantiation, we introduce Least-Recently-Updated MHM (MHM-LRU), which guarantees uniform head utilization with zero additional token overhead.
  arXiv  Detail & Related papers  (2026-07-01T22:38:54Z)
• Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory [50.857546269660276]
  We introduce SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving large language models (LLMs) memory.<n>Rather than focusing only on final performance, SeqMem-Eval evaluates how memory states evolve, generalize, consolidate experience, and retain useful information during sequential inference.
  arXiv  Detail & Related papers  (2026-05-14T20:15:22Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment [54.72137309071243]
  AlpsBench comprises 2,500 long-term interaction sequences curated from WildChat, paired with human-verified structured memories.<n>We define four pivotal tasks - personalized information extraction, updating, retrieval, and utilization - and establish protocols to evaluate the entire lifecycle of memory management.<n>Our benchmarking of frontier LLMs and memory-centric systems reveals that: (i) models struggle to reliably extract latent user traits; (ii) memory updating faces a performance ceiling even in the strongest models; (iii) retrieval accuracy declines sharply in the presence of large distractor pools; and (iv) while explicit memory mechanisms improve recall, they do not
  arXiv  Detail & Related papers  (2026-03-09T11:06:19Z)
• MemPO: Self-Memory Policy Optimization for Long-Horizon Agents [52.00646524941419]
  Existing methods typically introduce the external memory module and look up the relevant information from the stored memory.<n>We propose the self-memory policy optimization algorithm (MemPO), which enables the agent to autonomously summarize and manage their memory.<n>MemPO achieves absolute F1 score gains of 25.98% over the base model and 7.1% over the previous SOTA baseline, while reducing token usage by 67.58% and 73.12%.
  arXiv  Detail & Related papers  (2026-02-28T14:43:02Z)
• How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants [25.83552447206606]
  Large language model (LLM)-powered assistants have recently integrated memory mechanisms that record user preferences, leading to more personalized and user-aligned responses.<n>We develop RPEval, a benchmark comprising a personalized intent reasoning dataset and a multi-granularity evaluation protocol.<n>RPEval reveals the widespread phenomenon of irrational personalization in existing LLMs and, through error pattern analysis, illustrates its negative impact on user experience.<n>We introduce RP-Reasoner, which treats memory utilization as a pragmatic reasoning process, enabling the selective integration of personalized information.
  arXiv  Detail & Related papers  (2026-01-23T10:19:48Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.