MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models

• URL: http://arxiv.org/abs/2601.11969v2
• Date: Sat, 24 Jan 2026 06:24:16 GMT
• Title: MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models
• Authors: Zecheng Tang, Baibei Ji, Ruoxi Sun, Haitian Wang, WangJie You, Zhang Yijun, Wenpeng Zhu, Ji Qi, Juntao Li, Min Zhang,
• Abstract summary: We introduce MemoryRewardBench, the first benchmark to systematically study the ability of reward models to evaluate memory quality.<n> Evaluations on 13 cutting-edge RMs indicate a diminishing performance gap between open-source and proprietary models.
• Score: 40.965722377085456
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing works increasingly adopt memory-centric mechanisms to process long contexts in a segment manner, and effective memory management is one of the key capabilities that enables large language models to effectively propagate information across the entire sequence. Therefore, leveraging reward models (RMs) to automatically and reliably evaluate memory quality is critical. In this work, we introduce MemoryRewardBench, the first benchmark to systematically study the ability of RMs to evaluate long-term memory management processes. MemoryRewardBench covers both long-context comprehension and long-form generation tasks, featuring 10 distinct settings with different memory management patterns, with context length ranging from 8K to 128K tokens. Evaluations on 13 cutting-edge RMs indicate a diminishing performance gap between open-source and proprietary models, with newer-generation models consistently outperforming their predecessors regardless of parameter count. We further expose the capabilities and fundamental limitations of current RMs in evaluating LLM memory management across diverse settings.

Related papers

• UMEM: Unified Memory Extraction and Management Framework for Generalizable Memory [46.87954895079213]
  Self-evolving memory serves as the trainable parameters for Large Language Models (LLMs)<n>Existing methods predominately optimize memory management while treating memory extraction as a static process.<n>We propose Unified Memory Extraction and Management (UMEM) to jointly optimize a Large Language Model to simultaneous extract and manage memories.
  arXiv  Detail & Related papers  (2026-02-11T08:58:41Z)
• Mem-T: Densifying Rewards for Long-Horizon Memory Agents [23.19373149519922]
  We introduce Mem-T, an autonomous memory agent that interfaces with a lightweight hierarchical memory database to perform dynamic updates and multi-turn retrieval over streaming inputs.<n>We also propose MoT-GRPO, a tree-guided reinforcement learning framework that transforms sparse terminal feedback into dense, step-wise supervision via memory operation tree backpropagation and hindsight credit assignment.
  arXiv  Detail & Related papers  (2026-01-30T14:23:33Z)
• Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents [76.76004970226485]
  Long-term memory is a critical capability for multimodal large language model (MLLM) agents.<n>Mem-Gallery is a new benchmark for evaluating multimodal long-term conversational memory in MLLM agents.
  arXiv  Detail & Related papers  (2026-01-07T02:03:13Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• MemLoRA: Distilling Expert Adapters for On-Device Memory Systems [71.32550994522738]
  Memory-augmented Large Language Models (LLMs) have demonstrated remarkable consistency during dialogues.<n>MemLoRA is a novel memory system that integrates small Vision-Language Models.<n>VLM-integrated MemLoRA-V shows massive improvements in caption-based approaches.
  arXiv  Detail & Related papers  (2025-12-04T12:56:30Z)
• Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning [89.55738101744657]
  Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless.<n>We present Memory-R1, a reinforcement learning framework that equips LLMs with the ability to actively manage and utilize external memory.
  arXiv  Detail & Related papers  (2025-08-27T12:26:55Z)
• Multiple Memory Systems for Enhancing the Long-term Memory of Agent [9.43633399280987]
  Existing methods, such as MemoryBank and A-MEM, have poor quality of stored memory content.<n>We have designed a multiple memory system inspired by cognitive psychology theory.
  arXiv  Detail & Related papers  (2025-08-21T06:29:42Z)
• MemOS: A Memory OS for AI System [116.87568350346537]
  Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI)<n>Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.<n>MemOS is a memory operating system that treats memory as a manageable system resource.
  arXiv  Detail & Related papers  (2025-07-04T17:21:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.