HiMPO: Hindsight-Informed Memory Policy Optimization for Less-Entangled Credit in Long-Horizon Agents

• URL: http://arxiv.org/abs/2606.16285v1
• Date: Mon, 15 Jun 2026 06:49:18 GMT
• Title: HiMPO: Hindsight-Informed Memory Policy Optimization for Less-Entangled Credit in Long-Horizon Agents
• Abstract summary: HiMPO is a Hindsight-Informed Memory Policy Optimization framework for assigning less-entangled credit to memory-writing actions.<n>We show that HiMPO reduces blame leakage from tool-induced errors and improves attribution fidelity of memory updates.
• Score: 10.128343847630552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon agents rely on memory mechanisms to compress interaction history, but optimizing memory writing faces a distinct credit assignment challenge: a memory update may be rewarded or penalized due to downstream tool failures, noisy observations, or reasoning errors rather than its own contribution. This causally entangled credit can lead agents to discard useful evidence or preserve irrelevant information. We propose HiMPO, a Hindsight-Informed Memory Policy Optimization framework for assigning less-entangled credit to memory-writing actions in long-horizon agents. HiMPO first estimates the local utility of a memory update by comparing the task-relevant information recoverable from the previous and updated memories under the same pre-write state. It then uses hindsight relevance as a bounded retrospective filter that attenuates memory credit when local utility is not supported by the target outcome. The resulting memory-specific advantage is applied only to memory tokens, while trajectory-level rewards optimize the rest of the agent behavior. Across judge-based open-domain tasks and objective compressive-memory QA, HiMPO improves over strong memory-based and RL-based baselines while preserving compressed-context efficiency. Controlled interventions further show that HiMPO reduces blame leakage from tool-induced errors and improves attribution fidelity of memory updates.

Related papers

• Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents [97.19033877543522]
  We present a controlled harness evaluation of memory substrates for memory-augmented agents.<n>We instrument 26 performance and efficiency metrics under a unified harness.<n>These findings motivate substrate routing as a necessary component of adaptive agent memory systems.
  arXiv  Detail & Related papers  (2026-08-15T03:29:48Z)
• EvoGraph-Mem: Failure-Aware Editable Graph Memory for Long-Term Language Agents [4.265255679736993]
  Existing memory-augmented agents mainly focus on storing and retrieving past experience.<n>Previously distilled insights can become outdated, over-generalized, or harmful under new task contexts.<n>We propose a failure-aware memory maintenance framework based on an editable insight graph.
  arXiv  Detail & Related papers  (2026-08-03T12:35:44Z)
• Mechanistic Attention Guidance for Agent Memory Refinement [32.57216046158823]
  We show that retrieval-head attention provides a mechanistic signal for revealing segment-level memory utilization.<n>We propose Attention-Guided Memory Refinement (AGMR), a framework that uses utilization patterns revealed by attention to guide targeted segment-level memory updates.
  arXiv  Detail & Related papers  (2026-07-20T07:17:50Z)
• MemSyco-Bench: Benchmarking Sycophancy in Agent Memory [32.49512092540906]
  MemSyco-Bench is a benchmark for evaluating memory-induced sycophancy in agent systems.<n>MemSyco-Bench measures when memory should influence a decision and how valid memory should be used.
  arXiv  Detail & Related papers  (2026-07-01T15:30:33Z)
• MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery [58.64329475683699]
  We argue that long-term memory should be evaluated as an auditable post-interaction artifact.<n>We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users.<n>We find that successful assistance and recoverable memory behave as distinct capabilities.
  arXiv  Detail & Related papers  (2026-06-23T13:52:46Z)
• Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks [69.19366746169906]
  We present PRISM, a transformer-based architecture for visuomotor policies to use short-term memory.<n>Piracy filters retrieved information to suppress irrelevant details, improving performance.<n> hierarchical architecture compresses local information into compact tokens, improving its compute and memory footprint.
  arXiv  Detail & Related papers  (2026-06-15T03:49:23Z)
• REAL: A Reasoning-Enhanced Graph Framework for Long-Term Memory Management of LLMs [17.526686616588794]
  Large Language Models (LLMs) are increasingly expected to interact with users over long time horizons.<n>LLMs cannot retain all past interactions, making long-term memory management essential for storing, updating, and retrieving historical information.<n> REAL constructs long-term conversational memory as a temporal and confidence-aware directed property graph.
  arXiv  Detail & Related papers  (2026-06-09T10:53:10Z)
• AdMem: Advanced Memory for Task-solving Agents [13.611633561917513]
  Large Language Models (LLMs) show promise as tool-using agents but remain limited in long-horizon tasks that require remembering, organizing, and reusing knowledge.<n>We introduce a unified and automatic memory framework that integrates semantic, episodic, and procedural memory in a bi-level design combining short-term and long-term stores.
  arXiv  Detail & Related papers  (2026-06-05T00:11:57Z)
• Beyond Similarity: Trustworthy Memory Search for Personal AI Agents [25.265839311088516]
  We study memory search as a trust boundary in personal AI agents.<n>MemGate is a lightweight and deployable memory plug-in for trustworthy memory search.
  arXiv  Detail & Related papers  (2026-06-04T11:54:29Z)
• Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents [51.30250860677378]
  We propose MRAgent, a framework that combines an associative memory graph with an active reconstruction mechanism.<n>Running on a Cue-Tag-Content graph, our active reconstruction mechanism integrates reasoning directly into memory access.<n>Experiments on the LoCoMo benchmark and LongMemEval benchmark demonstrate significant improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T11:29:46Z)
• MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models [56.31411457917676]
  We introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.<n>Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods.
  arXiv  Detail & Related papers  (2026-05-27T06:04:19Z)
• Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents [0.33842793760651557]
  We propose a causal memory-selection technique that estimates how candidate memories affect the model's answer under controlled interventions.<n>Results show that CMI achieves a stronger balance between answer quality and robustness to misleading memory, suggesting that reliable long-term memory requires selecting context based on causal usefulness rather than relevance alone.
  arXiv  Detail & Related papers  (2026-05-17T20:21:55Z)
• From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents [38.52713500119118]
  We introduce Memora, a long-term memory benchmark spanning weeks to months long user conversations.<n>The benchmark evaluates three memory-grounded tasks: remembering, reasoning, and recommending.<n>Forgetting-Aware Memory Accuracy (FAMA) is a metric that penalizes reliance on obsolete or invalidated memory.
  arXiv  Detail & Related papers  (2026-04-21T21:31:01Z)
• MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution [52.29303869050117]
  Memory-augmented LLM agents maintain external memory banks to support long-horizon interaction.<n>MemMA is a plug-and-play multi-agent framework that coordinates the memory cycle along both the forward and backward paths.
  arXiv  Detail & Related papers  (2026-03-19T10:15:59Z)
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management [63.48041801851891]
  Fine-Mem is a unified framework designed for fine-grained feedback alignment.<n> Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-01-13T11:06:17Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.