Belief Memory: Agent Memory Under Partial Observability

• URL: http://arxiv.org/abs/2605.05583v2
• Date: Fri, 08 May 2026 02:08:49 GMT
• Title: Belief Memory: Agent Memory Under Partial Observability
• Abstract summary: We propose BeliefMem, which shifts the memory paradigm from committing to a single conclusion per observation to retaining multiple candidate conclusions with their probabilities.<n> BeliefMem preserves the uncertainty that the deterministic paradigm discards, enabling the agent to act with high confidence.<n> Empirical evaluations on LoCoMo and ALFWorld benchmarks show that, even with limited data, BeliefMem achieves the best average performance.
• Score: 56.41506249481312
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM agents that operate over long context depend on external memory to accumulate knowledge over time. However, existing methods typically store each observation as a single deterministic conclusion (e.g., inferring "API~X failed" from temporary errors), even though such observations are inherently partial and potentially ambiguous. By committing to one conclusion and discarding uncertainty, these methods introduce self-reinforcing error: the agent acts on the stored conclusion, never revisits alternatives, and reinforces the conclusion over time. To address this issue, we propose BeliefMem, which shifts the memory paradigm from committing to a single conclusion per observation to retaining multiple candidate conclusions with their probabilities. Concretely, BeliefMem stores the candidate conclusions as separate memory entries, each carrying a probability that is updated via Noisy-OR rules as new observations arrive. At retrieval, all candidates surface together with their probabilities, keeping alternatives visible to the agent. Since each conclusion in memory retains its probability, BeliefMem preserves the uncertainty that the deterministic paradigm discards, enabling the agent to act with high confidence on well-evidenced knowledge while retaining the capacity to update its confidence when new evidence arrives. Empirical evaluations on LoCoMo and ALFWorld benchmarks show that, even with limited data, BeliefMem achieves the best average performance, remarkably outperforming well-known baselines. More broadly, such probabilistic memory produces substantial gains and explores a new direction for agent memory in partially observable environments.

Related papers

• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory [1.0152838128195467]
  Memory Contagion -- the cross-temporal propagation of evaluator bias through agent memory.<n>We show that when agents are trained or guided by biased evaluators, their experiences become biased.<n>Our findings expose a critical but contingent vulnerability in current agent memory designs.
  arXiv  Detail & Related papers  (2026-06-22T11:43:37Z)
• Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents [13.846550145896314]
  Memory-augmented LLM agents tackle complex long-horizon tasks by summarizing interaction trajectories into compact memory.<n>Existing approaches typically train these memory policies using outcome-based reinforcement learning, failing to localize where intermediate memory quality degrades.<n>We argue that memory optimization should focus not merely on trajectory-level success, but on the clarity of the belief induced by intermediate summaries.
  arXiv  Detail & Related papers  (2026-05-28T16:17:19Z)
• WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction [72.1620416874118]
  Multimodal large language models are increasingly deployed as long-horizon agents.<n>Existing benchmarks measure recall over static dialogue, collapse memory into a single end-of-task accuracy, and reduce visual observations to captions.<n>We formulate multimodal agent memory as an Action-World Interaction Loop with an observable four-stage lifecycle.
  arXiv  Detail & Related papers  (2026-05-28T04:27:20Z)
• Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory [50.857546269660276]
  We introduce SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving large language models (LLMs) memory.<n>Rather than focusing only on final performance, SeqMem-Eval evaluates how memory states evolve, generalize, consolidate experience, and retain useful information during sequential inference.
  arXiv  Detail & Related papers  (2026-05-14T20:15:22Z)
• STALE: Can LLM Agents Know When Their Memories Are No Longer Valid? [5.361950931863979]
  Large Language Model (LLM) agents are increasingly expected to maintain coherent, long-term personalized memory.<n>Current benchmarks primarily measure static fact retrieval, overlooking the ability to revise stored beliefs when new evidence emerges.<n>We introduce STALE, a benchmark of 400 expert-validated conflict scenarios.<n>We present CUPMem, a prototype that strengthens write-time revision through structured state consolidation and propagation-aware search.
  arXiv  Detail & Related papers  (2026-05-07T16:31:15Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• The Model Agreed, But Didn't Learn: Diagnosing Surface Compliance in Large Language Models [12.743453676247903]
  Large Language Models (LLMs) internalize vast world knowledge as parametric memory, yet inevitably inherit the staleness and errors of their source corpora.<n> Knowledge editing offers a pivotal paradigm for surgically modifying memory without retraining.<n>We introduce a diagnostic framework that subjects models to discriminative self-assessment under in-context learning settings.
  arXiv  Detail & Related papers  (2026-04-07T15:20:41Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey [211.01908189012184]
  Memory, with hundreds of papers released this year, emerges as the critical solution to fill the utility gap.<n>We provide a unified view of foundation agent memory along three dimensions.<n>We then analyze how memory is instantiated and operated under different agent topologies.
  arXiv  Detail & Related papers  (2026-01-14T07:38:38Z)
• Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency [78.91846841708586]
  We show that even facts answered with perfect self-consistency can rapidly collapse under mild contextual interference.<n>We propose Neighbor-Consistency Belief (NCB), a structural measure of belief that evaluates response coherence across a conceptual neighborhood.<n>We also present Structure-Aware Training (SAT), which optimize context-invariant belief structure and reduces long-tail knowledge brittleness by approximately 30%.
  arXiv  Detail & Related papers  (2026-01-09T16:23:21Z)
• Finite Memory Belief Approximation for Optimal Control in Partially Observable Markov Decision Processes [1.614301262383079]
  We study finite memory belief approximation for partially observable (PO) optimal control (SOC) problems.<n>We develop a metric-based theory that directly relates information loss to control performance.
  arXiv  Detail & Related papers  (2026-01-06T16:05:20Z)
• Forgetting as a Feature: Cognitive Alignment of Large Language Models [39.146761527401424]
  We show that Large Language Models (LLMs) exhibit systematic forgetting of past information.<n> Drawing inspiration from human memory dynamics, we model LLM inference as a probabilistic memory process governed by exponential decay.<n>Building on these observations, we propose probabilistic memory prompting, a lightweight strategy that shapes evidence integration to mimic human-like memory decay.
  arXiv  Detail & Related papers  (2025-12-28T10:43:00Z)
• Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management [49.71055327567513]
  We argue that memory management should be viewed as a sequential decision-making problem under uncertainty.<n>Our contribution is not a new algorithm, but a principled reframing that clarifies the limitations of approaches.
  arXiv  Detail & Related papers  (2025-12-25T08:23:03Z)
• The Influence of Memory in Multi-Agent Consensus [0.8399688944263843]
  We propose a framework to study what we call emphmemory consensus protocol. We show that the employment of memory allows such processes to always converge, as well as, in some scenarios, such as cycles, converge faster. We provide a theoretical analysis of the probability of each option eventually winning such processes based on the initial opinions expressed by agents.
  arXiv  Detail & Related papers  (2021-05-10T20:59:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.