Quantifying Risk Under Evolving Uncertainty: Belief-Dependent Robustness for Safe Sequential Decision Making

• URL: http://arxiv.org/abs/2608.17574v1
• Date: Tue, 18 Aug 2026 09:36:19 GMT
• Title: Quantifying Risk Under Evolving Uncertainty: Belief-Dependent Robustness for Safe Sequential Decision Making
• Abstract summary: We propose RATTL (Risk-Adrial Total-Reward Learning), which ties caution to uncertainty.<n>We show the resulting planning problem is well posed under transience and compactness conditions.<n>RATTL targets safety for agents, including runtime-based systems, acting under uncertainty.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How cautious should an agent be while it is still learning its environment? We propose RATTL (Risk-Adversarial Total-Reward Learning), which ties caution to epistemic uncertainty: the agent holds a Bayesian posterior over unknown dynamics and plans against a Wasserstein ambiguity set whose radius is a monotone function of that posterior. The radius contracts with evidence, so behaviour interpolates continuously between worst-case robustness and risk-neutral total-reward maximization. The design follows the duality underlying the Entropic Value-at-Risk, which converts the choice of a risk level into the choice of an ambiguity radius. We show the resulting planning problem is well posed under transience and compactness conditions, and prove a Safety Sandwich: the RATTL value lies between the uninformed robust value and the full- knowledge optimum, with a gap that vanishes as the posterior concentrates. In a canonical binary-hazard instance, the induced criterion reduces to Conditional Value-at-Risk at a level set by the posterior entropy. A worked example shows the agent deferring the efficient action until a sharp identification threshold. RATTL targets runtime safety for agents, including LLM-based systems, acting under uncertainty.

Related papers

• Robust Risk Under Evolving Uncertainty: A Wasserstein Counterpart of the Entropic Value-at-Risk [0.0]
  We study the coherent risk measure it induces, the Wasserstein entropic value-at-risk.<n>Driving the transport radius by belief entropy then yields a closed-form robust dynamic-programming operator.
  arXiv  Detail & Related papers  (2026-08-19T16:22:48Z)
• Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability [79.08785366532287]
  We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability.<n>We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation.
  arXiv  Detail & Related papers  (2026-05-14T01:23:09Z)
• From Agent Failure Paths to Quantified Residual Risk: A Compositional Framework for Resilient Agentic AI [8.947753534117298]
  Agentic AI is crossing trust boundaries faster than current risk models can represent.<n>We propose a mechanism-to-magnitude pipeline for resilient agentic and embodied AI.<n>We show the framework on two contrasting scenarios a real-time warehouse robot and a governance-instrumented financial-services agent.
  arXiv  Detail & Related papers  (2026-04-28T23:40:24Z)
• RE-SAC: Disentangling aleatoric and epistemic risks in bus fleet control: A stable and robust ensemble DRL approach [19.34848029403215]
  Bus holding control is challenging due to traffic and passenger demand.<n>Standard actor-critic algorithms suffer from Q-value instability in volatile environments.<n>We propose a robust ensemble soft actor-critic framework to disentangle uncertainties.
  arXiv  Detail & Related papers  (2026-03-19T01:37:45Z)
• Uncertainty-Aware Jamming Mitigation with Active RIS: A Robust Stackelberg Game Approach [65.06640919319413]
  This paper investigates the jamming mitigation by leveraging an active reconfigurable intelligent surface (ARIS)<n>We adopt the Stackelberg game formulation to model the strategic interaction between the legitimate side and the adversary.<n>We first derive the optimal jamming policy as the follower's best response, which is then incorporated into the legitimate-side optimization for robust anti-jamming design.
  arXiv  Detail & Related papers  (2026-02-20T12:02:01Z)
• Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity [16.835098688159004]
  Tail-end risk measures such as static conditional value-at-risk (CVaR) are used in safety-critical applications to prevent rare, yet catastrophic events.<n>We develop risk-averse value and model-free Q-learning algorithms that rely on discretized augmented states.<n> Empirical results demonstrate that our algorithms successfully learn CVaR-sensitive policies and achieve effective performance-safety trade-offs.
  arXiv  Detail & Related papers  (2026-02-03T17:39:45Z)
• Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignment [49.2305683068875]
  We propose Risk-aware Stepwise Alignment (RSA), a novel alignment method that incorporates risk awareness into the policy optimization process.<n> RSA mitigates risks induced by excessive model shift away from a reference policy, and it explicitly suppresses low-probability yet high-impact harmful behaviors.<n> Experimental results demonstrate that our method achieves high levels of helpfulness while ensuring strong safety.
  arXiv  Detail & Related papers  (2025-12-30T14:38:02Z)
• From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks [54.25638567385662]
  Differential Privacy in machine learning is often interpreted as guarantees against membership inference.<n> translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem.<n>This paper bridges the critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated "from-scratch" attacks.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization [59.758009422067]
  We consider the problem of quantifying uncertainty over expected cumulative rewards in model-based reinforcement learning. We propose a new uncertainty Bellman equation (UBE) whose solution converges to the true posterior variance over values. We introduce a general-purpose policy optimization algorithm, Q-Uncertainty Soft Actor-Critic (QU-SAC) that can be applied for either risk-seeking or risk-averse policy optimization.
  arXiv  Detail & Related papers  (2023-12-07T15:55:58Z)
• Temporal Difference Uncertainties as a Signal for Exploration [76.6341354269013]
  An effective approach to exploration in reinforcement learning is to rely on an agent's uncertainty over the optimal policy. In this paper, we highlight that value estimates are easily biased and temporally inconsistent. We propose a novel method for estimating uncertainty over the value function that relies on inducing a distribution over temporal difference errors.
  arXiv  Detail & Related papers  (2020-10-05T18:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.