Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learning

• URL: http://arxiv.org/abs/2206.07915v1
• Date: Thu, 16 Jun 2022 04:38:50 GMT
• Title: Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learning
• Authors: Hejun Huang, Zhenglong Li, Dongkun Han
• Abstract summary: This work adopts control barrier functions over reinforcement learning, and proposes a compensated algorithm to completely maintain safety. Compared to quadratic programming based reinforcement learning methods, our sum-of-squares programming based reinforcement learning has shown its superiority.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safety guarantee is essential in many engineering implementations. Reinforcement learning provides a useful way to strengthen safety. However, reinforcement learning algorithms cannot completely guarantee safety over realistic operations. To address this issue, this work adopts control barrier functions over reinforcement learning, and proposes a compensated algorithm to completely maintain safety. Specifically, a sum-of-squares programming has been exploited to search for the optimal controller, and tune the learning hyperparameters simultaneously. Thus, the control actions are pledged to be always within the safe region. The effectiveness of proposed method is demonstrated via an inverted pendulum model. Compared to quadratic programming based reinforcement learning methods, our sum-of-squares programming based reinforcement learning has shown its superiority.

Related papers

• Safely Learning Controlled Stochastic Dynamics [61.82896036131116]
  We introduce a method that ensures safe exploration and efficient estimation of system dynamics.<n>After training, the learned model enables predictions of the system's dynamics and permits safety verification of any given control.<n>We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics.
  arXiv  Detail & Related papers  (2025-06-03T11:17:07Z)
• Leveraging Analytic Gradients in Provably Safe Reinforcement Learning [6.5301153208275675]
  Provably safe reinforcement learning is an active field of research that aims to provide such guarantees using safeguards.<n>We develop the first effective safeguard for analytic gradient-based reinforcement learning.
  arXiv  Detail & Related papers  (2025-06-02T13:35:03Z)
• Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning [57.84059344739159]
  "Shielding" is a popular technique to enforce safety inReinforcement Learning (RL) We propose a new permissibility-based framework to deal with safety and shield construction.
  arXiv  Detail & Related papers  (2024-05-29T18:00:21Z)
• Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey [11.180978323594822]
  Reinforcement learning is a powerful technique for developing new robot behaviors. It aims to incorporate safety considerations, enabling faster transfer to real robots and facilitating lifelong learning. One promising approach within safe reinforcement learning is the use of control barrier functions.
  arXiv  Detail & Related papers  (2024-04-22T22:52:14Z)
• Reinforcement Learning with Ensemble Model Predictive Safety Certification [2.658598582858331]
  unsupervised exploration prevents the deployment of reinforcement learning algorithms on safety-critical tasks. We propose a new algorithm that combines model-based deep reinforcement learning with tube-based model predictive control to correct the actions taken by a learning agent. Our results show that we can achieve significantly fewer constraint violations than comparable reinforcement learning methods.
  arXiv  Detail & Related papers  (2024-02-06T17:42:39Z)
• Approximate Model-Based Shielding for Safe Reinforcement Learning [83.55437924143615]
  We propose a principled look-ahead shielding algorithm for verifying the performance of learned RL policies. Our algorithm differs from other shielding approaches in that it does not require prior knowledge of the safety-relevant dynamics of the system. We demonstrate superior performance to other safety-aware approaches on a set of Atari games with state-dependent safety-labels.
  arXiv  Detail & Related papers  (2023-07-27T15:19:45Z)
• Adaptive Aggregation for Safety-Critical Control [3.1692938090731584]
  We propose an adaptive aggregation framework for safety-critical control. Our algorithm can achieve fewer safety violations while showing better data efficiency compared with several baselines.
  arXiv  Detail & Related papers  (2023-02-07T16:53:33Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions [35.9713619595494]
  Reinforcement Learning and continuous nonlinear control have been successfully deployed in multiple domains of complicated sequential decision-making tasks. Given the exploration nature of the learning process and the presence of model uncertainty, it is challenging to apply them to safety-critical control tasks. We propose a emphprovably efficient episodic safe learning framework for online control tasks.
  arXiv  Detail & Related papers  (2022-07-29T00:54:35Z)
• Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planning [4.2955354157580325]
  One of the key challenges to deep reinforcement learning (deep RL) is to ensure safety at both training and testing phases. We propose a novel technique of unsupervised action planning to improve the safety of on-policy reinforcement learning algorithms. Our results show that the proposed safety RL algorithm can achieve higher rewards compared with multiple baselines in both discrete and continuous control problems.
  arXiv  Detail & Related papers  (2021-09-29T10:26:29Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)
• Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems [81.7983463275447]
  Learning-based control algorithms require data collection with abundant supervision for training. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
  arXiv  Detail & Related papers  (2020-05-09T05:57:43Z)
• Cautious Reinforcement Learning with Logical Constraints [78.96597639789279]
  An adaptive safe padding forces Reinforcement Learning (RL) to synthesise optimal control policies while ensuring safety during the learning process. Theoretical guarantees are available on the optimality of the synthesised policies and on the convergence of the learning algorithm.
  arXiv  Detail & Related papers  (2020-02-26T00:01:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.