Learning-Enhanced Safeguard Control for High-Relative-Degree Systems: Robust Optimization under Disturbances and Faults

• URL: http://arxiv.org/abs/2501.15373v1
• Date: Sun, 26 Jan 2025 03:03:02 GMT
• Title: Learning-Enhanced Safeguard Control for High-Relative-Degree Systems: Robust Optimization under Disturbances and Faults
• Authors: Xinyang Wang, Hongwei Zhang, Shimin Wang, Wei Xiao, Martin Guay,
• Abstract summary: This paper aims to enhance system performance with safety guarantee in reinforcement learning-based optimal control problems. The concept of gradient similarity is proposed to quantify the relationship between the gradient of safety and the gradient of performance. gradient manipulation and adaptive mechanisms are introduced in the safe RL framework to enhance the performance with a safety guarantee.
• Score: 6.535600892275023
• License:
• Abstract: Merely pursuing performance may adversely affect the safety, while a conservative policy for safe exploration will degrade the performance. How to balance the safety and performance in learning-based control problems is an interesting yet challenging issue. This paper aims to enhance system performance with safety guarantee in solving the reinforcement learning (RL)-based optimal control problems of nonlinear systems subject to high-relative-degree state constraints and unknown time-varying disturbance/actuator faults. First, to combine control barrier functions (CBFs) with RL, a new type of CBFs, termed high-order reciprocal control barrier function (HO-RCBF) is proposed to deal with high-relative-degree constraints during the learning process. Then, the concept of gradient similarity is proposed to quantify the relationship between the gradient of safety and the gradient of performance. Finally, gradient manipulation and adaptive mechanisms are introduced in the safe RL framework to enhance the performance with a safety guarantee. Two simulation examples illustrate that the proposed safe RL framework can address high-relative-degree constraint, enhance safety robustness and improve system performance.

Related papers

• A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems [8.347548017994178]
  Safety and performance could be competing objectives, which makes their co-optimization difficult. We propose a state-constrained optimal control problem, where performance objectives are encoded via a cost function and safety requirements are imposed as state constraints. We demonstrate that the resultant value function satisfies a Hamilton-Jacobi-Bellman equation, which we approximate efficiently using a novel machine learning framework.
  arXiv  Detail & Related papers  (2025-02-16T09:46:17Z)
• Pareto Control Barrier Function for Inner Safe Set Maximization Under Input Constraints [50.920465513162334]
  We introduce the PCBF algorithm to maximize the inner safe set of dynamical systems under input constraints. We validate its effectiveness through comparison with Hamilton-Jacobi reachability for an inverted pendulum and through simulations on a 12-dimensional quadrotor system. Results show that the PCBF consistently outperforms existing methods, yielding larger safe sets and ensuring safety under input constraints.
  arXiv  Detail & Related papers  (2024-10-05T18:45:19Z)
• Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems [14.166970599802324]
  Optimal control methods provide solutions to safety-critical problems but easily become intractable. We propose a Reinforcement Learning-based Receding Horizon Control approach leveraging Model Predictive Control. We validate our method by applying it to the challenging automated merging control problem for Connected and Automated Vehicles.
  arXiv  Detail & Related papers  (2024-03-26T02:49:08Z)
• Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems [15.863561935347692]
  We develop provably safe and convergent reinforcement learning algorithms for control of nonlinear dynamical systems. Recent advances at the intersection of control and RL follow a two-stage, safety filter approach to enforcing hard safety constraints. We develop a single-stage, sampling-based approach to hard constraint satisfaction that learns RL controllers enjoying classical convergence guarantees.
  arXiv  Detail & Related papers  (2024-03-06T19:39:20Z)
• Safe Neural Control for Non-Affine Control Systems with Differentiable Control Barrier Functions [58.19198103790931]
  This paper addresses the problem of safety-critical control for non-affine control systems. It has been shown that optimizing quadratic costs subject to state and control constraints can be sub-optimally reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) We incorporate higher-order CBFs into neural ordinary differential equation-based learning models as differentiable CBFs to guarantee safety for non-affine control systems.
  arXiv  Detail & Related papers  (2023-09-06T05:35:48Z)
• Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach [1.8924647429604111]
  Barrier-Lyapunov Actor-Critic (BLAC) framework helps maintain the aforementioned safety and stability for the system. Additional backup controller is introduced in case the RL-based controller cannot provide valid control signals.
  arXiv  Detail & Related papers  (2023-04-08T16:48:49Z)
• Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments [84.3830478851369]
  We propose a safe reinforcement learning approach that can jointly learn the environment and optimize the control policy. Our approach can effectively enforce hard safety constraints and significantly outperform CMDP-based baseline methods in system safe rate measured via simulations.
  arXiv  Detail & Related papers  (2022-09-29T20:49:25Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Safe Model-Based Reinforcement Learning Using Robust Control Barrier Functions [43.713259595810854]
  An increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework.
  arXiv  Detail & Related papers  (2021-10-11T17:00:45Z)
• Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty [77.18483084440182]
  Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. We present a Gaussian Process (GP)-based approach to tackle the problem of model uncertainty in safety-critical controllers that use CBFs and CLFs.
  arXiv  Detail & Related papers  (2021-06-13T23:08:49Z)
• Learning Robust Hybrid Control Barrier Functions for Uncertain Systems [68.30783663518821]
  We propose robust hybrid control barrier functions as a means to synthesize control laws that ensure robust safety. Based on this notion, we formulate an optimization problem for learning robust hybrid control barrier functions from data. Our techniques allow us to safely expand the region of attraction of a compass gait walker that is subject to model uncertainty.
  arXiv  Detail & Related papers  (2021-01-16T17:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.