Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents

• URL: http://arxiv.org/abs/2606.10616v4
• Date: Thu, 18 Jun 2026 02:39:41 GMT
• Title: Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents
• Abstract summary: Long-horizon language agents accumulate observations, reasoning traces, and retrieved facts windows.<n>Existing systems treat retention as local and do not model long-term consequences under observability constraints.<n>We propose OSL-MR, a learning-augmented framework that enforces a strict separation between online-observable features and offline-available supervision.
• Score: 16.193335398478386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-horizon language agents accumulate observations, reasoning traces, and retrieved facts exceeding context windows, making memory retention a fundamental resource-allocation problem. Existing systems treat retention as local and do not model long-term consequences under observability constraints. To fill this gap, we formulate memory retention as a constrained stochastic optimization with budget feasibility, evidence utility, and delayed costs including miss, reacquisition, and stale penalties. We show this multi-step problem is NP-hard, making exact solution intractable. Moreover, deployment decisions must be made under partial observability. To address these challenges, we propose OSL-MR (Observability-Safe Learning for Memory Retention), a learning-augmented framework that enforces a strict separation between online-observable features and offline-available supervision. OSL-MR combines an evidence learner trained from realized evidence with a Mixed-Score heuristic that serves as a deployable online-safe baseline and an inductive prior. The policy learns query-conditioned evidence from interaction data and remains deployable under the same constraints. Experiments on LoCoMo and LongMemEval show OSL-MR outperforms recency-based, Generative Agents-style, and other heuristic baselines, especially under tight budgets. The Mixed-Score prior improves precision and recall, and sensitivity analysis shows robustness across cost settings. On small solvable instances, single-step optimization is insufficient to anticipate future demand shifts, while OSL-MR stays significantly closer to the dynamic-programming optimum, confirming the necessity of the sequential formulation and reinforcing our learning-guided approximation. These results establish constrained stochastic optimization and optimization-guided learning as a principled foundation for memory management in long-horizon agents.

Related papers

• REAL: A Reasoning-Enhanced Graph Framework for Long-Term Memory Management of LLMs [17.526686616588794]
  Large Language Models (LLMs) are increasingly expected to interact with users over long time horizons.<n>LLMs cannot retain all past interactions, making long-term memory management essential for storing, updating, and retrieving historical information.<n> REAL constructs long-term conversational memory as a temporal and confidence-aware directed property graph.
  arXiv  Detail & Related papers  (2026-06-09T10:53:10Z)
• Uncertainty-Aware LLM-Guided Policy Shaping for Sparse-Reward Reinforcement Learning [2.384534878752428]
  We propose Uncertainty-Aware LLM-Guided Policy Shaping (ULPS), a framework that integrates a calibrated Large Language Model into the Reinforcement Learning training loop.<n>ULPS employs an A*-based oracle to synthesize optimal symbolic trajectories, which are used to fine-tune a BERT-based language model.<n>We observe consistent improvements in success rate, reward efficiency, and sample complexity over unguided, uncalibrated, and standard RL baselines.
  arXiv  Detail & Related papers  (2026-06-04T19:46:45Z)
• Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents [13.846550145896314]
  Memory-augmented LLM agents tackle complex long-horizon tasks by summarizing interaction trajectories into compact memory.<n>Existing approaches typically train these memory policies using outcome-based reinforcement learning, failing to localize where intermediate memory quality degrades.<n>We argue that memory optimization should focus not merely on trajectory-level success, but on the clarity of the belief induced by intermediate summaries.
  arXiv  Detail & Related papers  (2026-05-28T16:17:19Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models [64.2820121997882]
  MIITA is a Memory-Induced Inference-Time Adaptation framework for supervised CL under constrained storage.<n>It stores supervised experiences as compact correction-direction prototypes with semantic anchors, and retrieves them at inference time using semantic and uncertainty-based cues.<n>Experiments show that MIITA consistently improves final performance and mitigates forgetting under fixed memory budgets.
  arXiv  Detail & Related papers  (2026-05-20T03:03:04Z)
• Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Framework [18.46738022193615]
  Long-term memory has emerged as a foundational component of autonomous Large Language Model (LLM) agents.<n>critical concerns regarding memory governance, semantic drift, and privacy vulnerabilities have surfaced.<n>We propose the Stability and Safety-Governed Memory (SSGM) framework, a conceptual governance architecture.
  arXiv  Detail & Related papers  (2026-03-12T10:16:52Z)
• Adaptive Memory Admission Control for LLM Agents [9.04001220868675]
  We propose Adaptive Memory Admission Control (A-MAC), a framework that treats memory admission as a structured decision problem.<n>A-MAC decomposes memory value into five complementary and interpretable factors: future utility, factual confidence, semantic novelty, temporal recency, and content type prior.<n>A-MAC achieves a superior precision-recall tradeoff, improving F1 to 0.583 while reducing latency by 31% compared to state-of-the-art LLM-native memory systems.
  arXiv  Detail & Related papers  (2026-03-04T19:32:02Z)
• AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations [61.6579785305668]
  AMemGym is an interactive environment enabling on-policy evaluation and optimization for memory-driven personalization.<n>Our framework provides a scalable, diagnostically rich environment for advancing memory capabilities in conversational agents.
  arXiv  Detail & Related papers  (2026-03-02T15:15:11Z)
• Towards Sample-Efficient and Stable Reinforcement Learning for LLM-based Recommendation [56.92367609590823]
  Long Chain-of-Thought (Long CoT) reasoning has shown promise in Large Language Models (LLMs)<n>We argue that Long CoT is inherently ill-suited for the sequential recommendation domain.<n>We propose RISER, a novel Reinforced Item Space Exploration framework for Recommendation.
  arXiv  Detail & Related papers  (2026-01-31T10:02:43Z)
• Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking [78.69179041551014]
  We propose an information-theoretic reward modeling framework based on the Information Bottleneck principle.<n>We show that InfoRM filters out preference-irrelevant information to alleviate reward misgeneralization.<n>We also introduce IBL, a distribution-level regularization that penalizes such deviations, effectively expanding the optimization landscape.
  arXiv  Detail & Related papers  (2025-10-15T15:51:59Z)
• OFMU: Optimization-Driven Framework for Machine Unlearning [5.100622189286672]
  Large language models increasingly require the ability to unlearn specific knowledge, such as user requests, copyrighted materials, or outdated information.<n>We propose OFMU, a penalty-based bi-level optimization framework that explicitly prioritizes forgetting while preserving retention.<n>We show that OFMU consistently outperforms existing unlearning methods in both efficacy and retained utility.
  arXiv  Detail & Related papers  (2025-09-26T15:31:32Z)
• Escaping Stability-Plasticity Dilemma in Online Continual Learning for Motion Forecasting via Synergetic Memory Rehearsal [19.181540661354312]
  We propose synergetic memory rehearsal (SyReM) for DNN-based motion forecasting.<n>SyReM maintains a compact memory buffer to represent learned knowledge.<n>It employs an inequality constraint that limits in the average loss over the memory buffer.<n>SyReM significantly mitigates catastrophic forgetting in past scenarios while improving forecasting accuracy in new ones.
  arXiv  Detail & Related papers  (2025-08-27T05:04:33Z)
• MemOS: A Memory OS for AI System [116.87568350346537]
  Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI)<n>Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.<n>MemOS is a memory operating system that treats memory as a manageable system resource.
  arXiv  Detail & Related papers  (2025-07-04T17:21:46Z)
• Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models [14.321060805197874]
  Large Language Models (LLMs) deployed in real-world settings increasingly face the need to unlearn sensitive, outdated, or proprietary information.<n>Existing unlearning methods formulate forgetting and retention as a regularized trade-off, combining both objectives into a single scalarized loss.<n>We propose a new formulation of LLM unlearning as a constrained optimization problem: forgetting is enforced via a novel logit-margin flattening loss.
  arXiv  Detail & Related papers  (2025-06-05T17:55:23Z)
• TACO: Think-Answer Consistency for Optimized Long-Chain Reasoning and Efficient Data Learning via Reinforcement Learning in LVLMs [50.820065021136024]
  DeepSeek R1 has significantly advanced complex reasoning for large language models (LLMs)<n>Recent methods have attempted to replicate R1's reasoning capabilities in multimodal settings.<n>We propose TACO, a novel reinforcement learning algorithm for visual reasoning.
  arXiv  Detail & Related papers  (2025-05-27T06:30:48Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.