Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty

• URL: http://arxiv.org/abs/2605.11436v1
• Date: Tue, 12 May 2026 02:37:04 GMT
• Title: Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty
• Abstract summary: We introduce Agent-BRACE: Agent Belief state Representation via Abstraction and Confidence Estimation.<n>It is a method that decouples an LLM agent into a belief state model and a policy model, jointly optimized via reinforcement learning.<n>It achieves an average absolute improvement of +14.5% across long-horizon, partially observable language environments.
• Score: 70.43119366710778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly deployed on long-horizon tasks in partially observable environments, where they must act while inferring and tracking a complex environment state over many steps. This leads to two challenges: partial observability requires maintaining uncertainty over unobserved world attributes, and long interaction history causes context to grow without bound, diluting task-relevant information. A principled solution to both challenges is a belief state: a posterior distribution over environment states given past observations and actions, which compactly encodes history for decision making regardless of episode length. In LLM agents, however, the open-ended nature of text makes it unclear how to represent such a distribution. Therefore, we introduce Agent-BRACE: Agent Belief state Representation via Abstraction and Confidence Estimation, a method that decouples an LLM agent into a belief state model and a policy model, jointly optimized via reinforcement learning. The belief state model produces a structured approximation of the belief distribution: a set of atomic natural language claims about the environment, each annotated with an ordinal verbalized certainty label ranging from certain to unknown. The policy model conditions on this compact, structured approximate belief rather than the full history, learning to select actions under explicit uncertainty. Across long-horizon, partially observable embodied language environments, Agent-BRACE achieves an average absolute improvement of +14.5% (Qwen2.5-3B-Instruct) and +5.3% (Qwen3-4B-Instruct), outperforming strong RL baselines while maintaining a near-constant context window independent of episode length. Further analysis shows that the learned belief becomes increasingly calibrated over the course of an episode as evidence accumulates.

Related papers

• Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability [0.0]
  Large language model agents produce fluent action sequences across a wide range of tasks, yet they fail in characteristic ways once the environment becomes partially observable.<n>An LLM agent, as commonly deployed, is a history-conditioned policy with no explicit belief over hidden state.<n>We propose an architectural fix. The Belief-State Engine (BSE) is an inference module placed outside the LLM.
  arXiv  Detail & Related papers  (2026-09-09T11:07:47Z)
• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents [75.69180947909148]
  We propose a trajectory-level UQ framework for large language model (LLM) agents.<n>RuPA represents an execution history as a directed trajectory graph in which reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges.<n>We evaluate RUPA on representative agent benchmarks, including $$-2, Terminal-Bench-2, and GAIA, using 6 open-source LLMs spanning multiple model families.
  arXiv  Detail & Related papers  (2026-08-17T01:40:14Z)
• Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs [51.56484100374058]
  Large Language Models (LLMs) demonstrate remarkable multilingual fluency, but their internal knowledge representations remain disproportionately biased toward high-resource languages.<n>We investigate whether these biases can be mitigated at inference time, forcing an English-prompted model to answer as if it were queried in target languages.<n>Experiments on Gemma 3 12B Instruct reveal persona prompting to be the strongest overall intervention, balancing efficacy, safety, and out-of-domain generalization.
  arXiv  Detail & Related papers  (2026-07-21T16:15:05Z)
• Ask the World Before Acting: Environment Probing for Calibrated Agent World Models [2.571318992830046]
  Language agents acting over long horizons must maintain beliefs about tool states, object locations, graph edges, and subgoal dependencies.<n>We formulate environment probing as a budgeted decision problem for structured agent world models.<n>We introduce EnvProbe, a simple scoring policy that combines task criticality, staleness, verbalized uncertainty, and dependency role.
  arXiv  Detail & Related papers  (2026-06-30T09:45:49Z)
• Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation [38.81368744366676]
  Large language models (LLMs) define a distribution over text, which can be viewed as a probabilistic representation of uncertainty.<n>We propose Belief-Augmented Generation (BAG): grounding LLMs in their own belief state via the prompt and letting them reason over these K samples to decide on a conversational strategy.<n>BAG improves QA accuracy across six models and yields strategy decisions more faithful to the belief state than prompt-only baselines.
  arXiv  Detail & Related papers  (2026-05-25T13:29:02Z)
• Why We Need World Models for AGI: Where LLMs Fail and How World Models May Outperform [1.6746114653388389]
  Large language models achieve strong performance in language generation and knowledge-intensive tasks.<n>We argue that these limitations may arise from an objective-level mismatch between sequence prediction and reasoning over latent environment dynamics.<n>We introduce Latent Dynamics Inference (LDI), a conceptual perspective that interprets language and multimodal observations as partial evidence of underlying transition dynamics.
  arXiv  Detail & Related papers  (2026-05-13T12:55:34Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• The Context Gathering Decision Process: A POMDP Framework for Agentic Search [38.92972416925679]
  Large Language Model (LLM) agents are deployed in complex environments.<n>Without explicit infrastructure, an agent's working memory can degrade into lossy representations of the search state.<n>We formalize this challenge as the Context Gathering Decision Process (CGDP)<n>We derive two plug-and-play interventions for iterative LLM agents.
  arXiv  Detail & Related papers  (2026-05-07T23:45:07Z)
• Learning Uncertainty from Sequential Internal Dispersion in Large Language Models [52.29267172760918]
  Uncertainty estimation is a promising approach to detect hallucinations in large language models.<n>Recent approaches commonly depend on model internal states to estimate uncertainty.<n>We present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework.
  arXiv  Detail & Related papers  (2026-04-17T06:31:29Z)
• Beyond Preset Identities: How Agents Form Stances and Boundaries in Generative Societies [28.436766185842767]
  This paper proposes a novel mixed-methods framework combining computational virtual ethnography with quantitative socio-cognitive profiling.<n>It formalizes three new metrics: Innate Value Bias (IVB), Persuasion Sensitivity, and Trust-Action Decoupling (TAD)<n>Findings expose the fragility of static prompt engineering, providing a methodological and quantitative foundation for dynamic alignment in human-agent hybrid societies.
  arXiv  Detail & Related papers  (2026-03-24T16:38:46Z)
• LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth [32.1520194112537]
  Large language models (LLMs) are increasingly capable of carrying out long-running, real-world tasks.<n>As the amount of context grows, their reliability often deteriorates, a phenomenon known as "context rot"
  arXiv  Detail & Related papers  (2026-02-08T13:20:39Z)
• ProAct: Agentic Lookahead in Interactive Environments [56.50613398808361]
  ProAct is a framework that enables agents to internalize accurate lookahead reasoning through a two-stage training paradigm.<n>We introduce Grounded LookAhead Distillation (GLAD), where the agent undergoes supervised fine-tuning on trajectories derived from environment-based search.<n>We also propose the Monte-Carlo Critic (MC-Critic), a plug-and-play auxiliary value estimator designed to enhance policy-gradient algorithms.
  arXiv  Detail & Related papers  (2026-02-05T05:45:16Z)
• Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency [78.91846841708586]
  We show that even facts answered with perfect self-consistency can rapidly collapse under mild contextual interference.<n>We propose Neighbor-Consistency Belief (NCB), a structural measure of belief that evaluates response coherence across a conceptual neighborhood.<n>We also present Structure-Aware Training (SAT), which optimize context-invariant belief structure and reduces long-tail knowledge brittleness by approximately 30%.
  arXiv  Detail & Related papers  (2026-01-09T16:23:21Z)
• Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games [2.357397994148727]
  Large Reasoning Models (LRMs) undergo a collapse in performance on solving puzzles beyond certain perplexity thresholds.<n>We provide a large language model (LLM) with an environment interface for Tower of Hanoi problems.<n>We observe that access to an environment interface does not delay or eradicate performance collapse.
  arXiv  Detail & Related papers  (2025-10-12T23:48:16Z)
• Invariant Causal Prediction for Block MDPs [106.63346115341862]
  Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. We propose a method of invariant prediction to learn model-irrelevance state abstractions (MISA) that generalize to novel observations in the multi-environment setting.
  arXiv  Detail & Related papers  (2020-03-12T21:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.