R^2-Mem: Reflective Experience for Memory Search

• URL: http://arxiv.org/abs/2605.13486v1
• Date: Wed, 13 May 2026 13:09:36 GMT
• Title: R^2-Mem: Reflective Experience for Memory Search
• Abstract summary: R2-Mem is a reflective experience framework for memory search systems.<n>In offline stage, a Learner-guided Evaluator scores low- and high-quality steps in historical trajectories.<n>During online inference, the retrieved experience will guide future search actions to avoid repeated mistakes.
• Score: 25.798601679871425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep search has recently emerged as a promising paradigm for enabling agents to retrieve fine-grained historical information without heavy memory pre-managed. However, existing deep search agents for memory system repeat past error behaviors because they fail to learn from the prior high- and low-quality search trajectories. To address this limitation, we propose R^2-Mem, a reflective experience framework for memory search systems. In the offline stage, a Rubric-guided Evaluator scores low- and high-quality steps in historical trajectories, and a self-Reflection Learner distills the corresponding abstract experience. During the online inference, the retrieved experience will guide future search actions to avoid repeated mistakes and maintain high-quality behaviors. Extensive experiments demonstrate that R^2-Mem consistently improves both effectiveness and efficiency over strong baselines, improving F1 scores by up to 22.6%, while reducing token consumption by 12.9% and search iterations by 20.2%. These results verify that R^2-Mem provides a RL-free and low-cost solution for self-improving LLM agents.

Related papers

• Exploratory and Assimilating Reflection: Reflective Recall Cycle for Long-term Memory [3.9297698032556596]
  LLM-based autonomous agents require external memory to overcome their statelessness and limited context window.<n>We propose Exploratory-Assimilating Reflection (EAR), a framework for high initial retrieval performance and sample-efficient adaptation.
  arXiv  Detail & Related papers  (2026-07-20T12:30:17Z)
• Agent Reinforcement Learning via Pivotal-Aware Self-Feedback Retry [14.02751159295063]
  We propose PivoARL, a self-feedback retry framework for experience exploitation in large language model (LLM) agents.<n>PivoARL identifies the pivotal erroneous turn through structured reflection and performs local retry only from the corresponding pivotal state.<n>We show that PivoARL achieves significant improvements on Pass@2/3 across all tasks, with an average gain of about 11.5% over MetaRL.
  arXiv  Detail & Related papers  (2026-07-04T04:45:05Z)
• Evoking User Memory: Personalizing LLM via Recollection-Familiarity Adaptive Retrieval [59.295767860331004]
  RF-Mem is a familiarity uncertainty-guided dual-path memory retriever.<n>It embeds human-like dual-process recognition into the retriever.<n>It consistently outperforms one-shot retrieval and full-context reasoning under fixed budget and latency constraints.
  arXiv  Detail & Related papers  (2026-03-10T06:31:44Z)
• RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback [54.39884046754265]
  RetroAgent is an online RL framework that enables agents to master complex interactive environments.<n>Our experiments show that RetroAgent achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-03-09T16:23:33Z)
• LightSearcher: Efficient DeepSearch via Experiential Memory [23.338677838845]
  We propose an efficient reinforcement learning framework that balances accuracy and efficiency in DeepSearch paradigms.<n>Experiments on four multi-hop QA benchmarks show that LightSearcher maintains accuracy comparable to SOTA baseline ReSearch.
  arXiv  Detail & Related papers  (2025-12-07T04:29:52Z)
• Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings [70.26204343623215]
  ColPali/ColQwen2 encodes each page into multiple patch-level embeddings and leads to excessive memory usage.<n>This empirical study investigates methods to reduce patch embeddings per page at minimum performance degradation.
  arXiv  Detail & Related papers  (2025-06-05T13:06:01Z)
• How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior [65.70584076918679]
  Memory is a critical component in large language model (LLM)-based agents.<n>This paper studies how memory management choices impact the LLM agents' behavior, especially their long-term performance.
  arXiv  Detail & Related papers  (2025-05-21T22:35:01Z)
• An Early FIRST Reproduction and Improvements to Single-Token Decoding for Fast Listwise Reranking [50.81324768683995]
  FIRST is a novel approach that integrates a learning-to-rank objective and leveraging the logits of only the first generated token. We extend the evaluation of FIRST to the TREC Deep Learning datasets (DL19-22), validating its robustness across diverse domains. Our experiments confirm that fast reranking with single-token logits does not compromise out-of-domain reranking quality.
  arXiv  Detail & Related papers  (2024-11-08T12:08:17Z)
• A simple but strong baseline for online continual learning: Repeated Augmented Rehearsal [13.075018350152074]
  Online continual learning (OCL) aims to train neural networks incrementally from a non-stationary data stream with a single pass through data. Rehearsal-based methods attempt to approximate the observed input distributions over time with a small memory and revisit them later to avoid forgetting. We provide theoretical insights on the inherent memory overfitting risk from the viewpoint of biased and dynamic empirical risk minimization.
  arXiv  Detail & Related papers  (2022-09-28T08:43:35Z)
• Retrieval-Augmented Reinforcement Learning [63.32076191982944]
  We train a network to map a dataset of past experiences to optimal behavior. The retrieval process is trained to retrieve information from the dataset that may be useful in the current context. We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
  arXiv  Detail & Related papers  (2022-02-17T02:44:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.