BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control

• URL: http://arxiv.org/abs/2605.16170v2
• Date: Tue, 19 May 2026 13:26:47 GMT
• Title: BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control
• Abstract summary: Real-world control systems operate under emphpiecewise stationary conditions, where dynamics remain stable for extended periods before undergoing regime changes.<n>Standard robust RL methods face a fundamental dilemma: a globally conservative policy wastes performance during stable periods, while a locally adaptive policy risks catastrophic failure when the regime changes undetected.<n>We propose textbfBAPR, which unifies Bayesian Online Change Detection with robust ensemble RL.
• Score: 19.34848029403215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world control systems frequently operate under \emph{piecewise stationary} conditions, where dynamics remain stable for extended periods before undergoing abrupt regime changes. Standard robust RL methods face a fundamental dilemma: a globally conservative policy wastes performance during stable periods, while a locally adaptive policy risks catastrophic failure when the regime changes undetected. We propose \textbf{BAPR} (Bayesian Amnesic Piecewise-Robust SAC), which unifies Bayesian Online Change Detection (BOCD) with robust ensemble RL. The BAPR operator -- a convex combination of mode-conditional Bellman operators weighted by a frozen belief distribution -- is a $γ$-contraction. A complementary counterexample, machine-verified in Lean~4, establishes a \emph{sharp boundary}: when beliefs depend on the Q-function, the contraction factor becomes $γ+ λΔ$ (where $Δ$ is the mode reward gap), and contraction fails exactly when $γ+ λΔ\geq 1$. We derive a \emph{component-wise} formal error budget for the abstract operator -- every component machine-verified -- bounding post-switch recovery; the budget applies to the abstract mode-mixture operator and inherits to the implemented shared-critic algorithm only through the frozen-parameter design intuition. All results are formally verified with no \texttt{sorry} (1,145 lines across 3 Lean~4 files, 22 machine-verified theorems). BOCD drives an adaptive conservatism mechanism: the policy becomes maximally conservative after detected change-points and smoothly relaxes as confidence grows, with detection delay $O(\log(1/δ))$. A context-conditioning module trained via RMDM loss provides mode-aware representations from simulator-provided mode IDs at training time and requires no mode labels at deployment.

Related papers

• Feedback-Coupled Memory Systems in Continuous Time [0.0]
  We show that memory dissipation must outpace feedback gain as a universal organizing principle in FCMS architecture.<n>This generalizes both the discrete FCMS stability condition $42 $ and CMGP's physical bifurcation threshold $_c = 1/K$.<n> Numerical simulation with $N=2$ agents and mean-field validation at $N=106$ confirm the stability threshold and the self-reinforcing coordination cascade that emerges when it is violated.
  arXiv  Detail & Related papers  (2026-06-24T14:03:34Z)
• Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents [0.0]
  Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change.<n>We propose the textbfInformational Viability Principle: governing an agent reduces to estimating a bound on unobserved risk.<n>textbfRiskGate instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest $z$-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot.
  arXiv  Detail & Related papers  (2026-04-27T16:46:15Z)
• A Nonasymptotic Theory of Gain-Dependent Error Dynamics in Behavior Cloning [4.822598110892847]
  Behavior cloning policies on position-controlled robots inherit the closed-loop response of the underlying PD controller.<n>We show that independent sub-Gaussian action errors propagate through the gain-dependent closed-loop dynamics.
  arXiv  Detail & Related papers  (2026-04-15T23:53:09Z)
• Optimistic Policy Learning under Pessimistic Adversaries with Regret and Violation Guarantees [6.310266319378214]
  Real-world decision-making systems operate in environments where state transitions depend on the agent's actions.<n>Standard Constrained MDP formulations assume the agent is the sole driver of state evolution.<n>We propose textbfRobust Hallucinated Constrained Upper-Confidence RL (textttRHC-UCRL), a model-based algorithm that maintains optimism over both agent and adversary policies.
  arXiv  Detail & Related papers  (2026-04-15T04:53:29Z)
• Tight Convergence Rates for Online Distributed Linear Estimation with Adversarial Measurements [66.94250413799232]
  We study mean estimation of a random vector $X$ in a distributed parameter-server-worker setup.<n>Key challenges are adversarial measurements and asynchrony.<n>Results provide a unified finite-time characterization of robustness, identifiability, and statistical efficiency in distributed linear estimation with adversarial workers.
  arXiv  Detail & Related papers  (2026-04-07T11:45:55Z)
• Soft-Label Governance for Distributional Safety in Multi-Agent Systems [4.726777092009554]
  Multi-agent AI systems exhibit emergent risks that no single agent produces in isolation.<n>We introduce SWARM, a simulation framework that replaces binary good/bad labels with emphsoft probabilistic labels.<n>In seven scenarios with five-seed replication, strict governance reduces welfare by over 40% without improving safety.
  arXiv  Detail & Related papers  (2026-03-19T00:56:09Z)
• Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning [61.07540493350384]
  Self-distillation (SD) is the process of retraining a student on a mixture of ground-truth and the teacher's own predictions.<n>We show that for any prediction risk, the optimally mixed student improves upon the ridge teacher for every regularization level.<n>We propose a consistent one-shot tuning method to estimate $star$ without grid search, sample splitting, or refitting.
  arXiv  Detail & Related papers  (2026-02-19T17:21:15Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• Efficient Policy Iteration for Robust Markov Decision Processes via Regularization [49.05403412954533]
  Robust decision processes (MDPs) provide a framework to model decision problems where the system dynamics are changing or only partially known. Recent work established the equivalence between texttts rectangular $L_p$ robust MDPs and regularized MDPs, and derived a regularized policy iteration scheme that enjoys the same level of efficiency as standard MDPs. In this work, we focus on the policy improvement step and derive concrete forms for the greedy policy and the optimal robust Bellman operators.
  arXiv  Detail & Related papers  (2022-05-28T04:05:20Z)
• Bayesian Bellman Operators [55.959376449737405]
  We introduce a novel perspective on Bayesian reinforcement learning (RL) Our framework is motivated by the insight that when bootstrapping is introduced, model-free approaches actually infer a posterior over Bellman operators, not value functions.
  arXiv  Detail & Related papers  (2021-06-09T12:20:46Z)
• Provably Efficient Safe Exploration via Primal-Dual Policy Optimization [105.7510838453122]
  We study the Safe Reinforcement Learning (SRL) problem using the Constrained Markov Decision Process (CMDP) formulation. We present an provably efficient online policy optimization algorithm for CMDP with safe exploration in the function approximation setting.
  arXiv  Detail & Related papers  (2020-03-01T17:47:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.