Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning

• URL: http://arxiv.org/abs/2605.26452v1
• Date: Tue, 26 May 2026 02:02:40 GMT
• Title: Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning
• Abstract summary: We propose a safety-filtered actor-critic framework that learns a finite-dimensional Koopman predictor from data.<n>The method achieves zero constraint violations on CartPole stabilization and tracking while matching or exceeding unconstrained SAC returns.<n>These results suggest that robust Koopman-CBF filters are a promising bridge between model-free RL and certifiable safety.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safe reinforcement learning (RL) for robotic systems requires policies that improve task performance while satisfying state and input constraints during both training and deployment. Control barrier functions (CBFs) provide a principled mechanism for enforcing forward invariance through minimally invasive safety filters, but their use in model-free RL is limited by the need for accurate dynamics and hand-designed barrier certificates. We propose Robust Koopman-CBF SAC, a safety-filtered actor--critic framework that learns a finite-dimensional Koopman predictor from data, constructs affine CBF constraints in the lifted space, and enforces them through a quadratic-program safety layer. To account for finite-dimensional Koopman approximation error, the CBF condition is tightened using a projected residual margin estimated from held-out rollout data. The critic is trained on the executed safe action, while the actor is regularized toward the Koopman-CBF feasible set, reducing dependence on the filter over training. Across safe-control benchmarks, the method achieves zero constraint violations on CartPole stabilization and tracking while matching or exceeding unconstrained SAC returns. On high-dimensional Safety Gymnasium locomotion tasks, the method reduces violations in some settings but also exposes important limitations of first-order velocity barriers and linear EDMD models, motivating high-order and multi-step Koopman-CBF extensions. These results suggest that robust Koopman-CBF filters are a promising bridge between model-free RL and certifiable safety, while clarifying the structural conditions under which such filters remain effective. All code is available at \href{https://github.com/DhruvKushwaha/Koopman-CBF-Soft-Actor-Critic}{Github Repository}.

Related papers

• Shield-Loco: Shielding Locomotion Policies with Predictive Safety Filtering [10.242244578585897]
  Large-scale offline safe learning is impractical for covering all edge cases.<n>We propose a predictive safety filter that post-hoc filters the nominal contact locations fed to the RL policy.<n>We validate the filter on a quadruped robot in dense, cluttered environments.
  arXiv  Detail & Related papers  (2026-06-05T11:59:43Z)
• Selective Safety Steering via Value-Filtered Decoding [54.87935112120107]
  Large language models (LLMs) are trained to align with human values, but their generations may still violate safety constraints.<n>Existing decoding-time steering methods often intervene unnecessarily, modifying generations that would have been safe under the base model.<n>We propose a new test-time steering method designed to reduce such unnecessary interventions while improving the safety of unsafe responses.
  arXiv  Detail & Related papers  (2026-05-14T12:13:08Z)
• Safety-Guaranteed Imitation Learning from Nonlinear Model Predictive Control for Spacecraft Close Proximity Operations [39.146761527401424]
  This paper presents a safety-guaranteed, runtime-efficient imitation learning framework for spacecraft close proximity control.<n>We leverage Control Barrier Functions (CBFs) for safety certificates and Control Lyapunov Functions (CLFs) for stability as unified design principles across data generation, training, and deployment.<n>A runtime analysis demonstrates real-time feasibility on a commercial off-the-shelf processor, supporting onboard deployment for safety-critical on-orbit servicing.
  arXiv  Detail & Related papers  (2026-03-19T13:47:18Z)
• Layered Safety: Enhancing Autonomous Collision Avoidance via Multistage CBF Safety Filters [18.386652442201665]
  This paper presents a general end-to-end framework for constructing robust and reliable layered safety filters.<n>Given a robot-centric point cloud, we begin by constructing an occupancy map which is used to synthesize a Poisson safety function (PSF)<n>The PSF is employed as a control barrier function (CBF) within two distinct safety filtering stages.
  arXiv  Detail & Related papers  (2026-02-27T22:19:45Z)
• BarrierSteer: LLM Safety via Learning Barrier Steering [83.12893815611052]
  BarrierSteer is a novel framework that formalizes safety by embedding learned non-linear safety constraints directly into the model's latent representation space.<n>We show that BarrierSteer substantially reduces adversarial success rates, decreases unsafe generations, and outperforms existing methods.
  arXiv  Detail & Related papers  (2026-02-23T18:19:46Z)
• How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints [21.03977709777739]
  We show that reachability value functions can be adapted to be control barrier functions (CBFs) for smooth optimization-based filtering.<n>We propose the LatentCBF, which addresses both challenges through gradient penalties that lead to smooth margin functions without additional labeling.<n>Experiments on simulated benchmarks and hardware with a vision-based manipulation policy demonstrate that LatentCBF enables smooth safety filtering.
  arXiv  Detail & Related papers  (2025-11-23T20:15:28Z)
• Safe Neural Control for Non-Affine Control Systems with Differentiable Control Barrier Functions [58.19198103790931]
  This paper addresses the problem of safety-critical control for non-affine control systems. It has been shown that optimizing quadratic costs subject to state and control constraints can be sub-optimally reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) We incorporate higher-order CBFs into neural ordinary differential equation-based learning models as differentiable CBFs to guarantee safety for non-affine control systems.
  arXiv  Detail & Related papers  (2023-09-06T05:35:48Z)
• A Multiplicative Value Function for Safe and Efficient Reinforcement Learning [131.96501469927733]
  We propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. We evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations.
  arXiv  Detail & Related papers  (2023-03-07T18:29:15Z)
• Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functions [5.571154223075409]
  This paper presents a method for safe and efficient reinforcement learning (RL) using disturbance observers (DOBs) and control barrier functions (CBFs) Our method does not involve model learning, and leverages DOBs to accurately estimate the pointwise value of the uncertainty, which is then incorporated into a robust CBF condition to generate safe actions. Simulation results on a unicycle and a 2D quadrotor demonstrate that the proposed method outperforms a state-of-the-art safe RL algorithm using CBFs and Gaussian processes-based model learning.
  arXiv  Detail & Related papers  (2022-11-30T18:49:53Z)
• Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty [77.18483084440182]
  Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. We present a Gaussian Process (GP)-based approach to tackle the problem of model uncertainty in safety-critical controllers that use CBFs and CLFs.
  arXiv  Detail & Related papers  (2021-06-13T23:08:49Z)
• Learning Control Barrier Functions from Expert Demonstrations [69.23675822701357]
  We propose a learning based approach to safe controller synthesis based on control barrier functions (CBFs) We analyze an optimization-based approach to learning a CBF that enjoys provable safety guarantees under suitable Lipschitz assumptions on the underlying dynamical system. To the best of our knowledge, these are the first results that learn provably safe control barrier functions from data.
  arXiv  Detail & Related papers  (2020-04-07T12:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.