Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

• URL: http://arxiv.org/abs/2108.06266v1
• Date: Fri, 13 Aug 2021 14:22:02 GMT
• Title: Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
• Authors: Lukas Brunke, Melissa Greeff, Adam W. Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, Angela P. Schoellig (University of Toronto Institute for Aerospace Studies, University of Toronto Robotics Institute, Vector Institute for Artificial Intelligence)
• Abstract summary: We review the recent advances made in using machine learning to achieve safe decision making under uncertainties. Our review includes: learning-based control approaches that safely improve performance by learning the uncertain dynamics. We highlight some of the open challenges that will drive the field of robot learning in the coming years.
• Score: 3.9258421820410225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The last half-decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. Our review includes: learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safety of a learned control policy. As data- and learning-based robot control methods continue to gain traction, researchers must understand when and how to best leverage them in real-world scenarios where safety is imperative, such as when operating in close proximity to humans. We highlight some of the open challenges that will drive the field of robot learning in the coming years, and emphasize the need for realistic physics-based benchmarks to facilitate fair comparisons between control and reinforcement learning approaches.

Related papers

• Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning [17.856459823003277]
  Safety is one of the key issues preventing the deployment of reinforcement learning techniques in real-world robots. In this paper, we bridge the gap by extending the safe exploration method, ATACOM, with learnable constraints.
  arXiv  Detail & Related papers  (2024-09-18T15:08:41Z)
• Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey [11.180978323594822]
  Reinforcement learning is a powerful technique for developing new robot behaviors. It aims to incorporate safety considerations, enabling faster transfer to real robots and facilitating lifelong learning. One promising approach within safe reinforcement learning is the use of control barrier functions.
  arXiv  Detail & Related papers  (2024-04-22T22:52:14Z)
• Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomy [7.915956857741506]
  Existing safety methods often neglect the robot's ability to learn and adapt at runtime, leading to overly conservative behavior. This paper proposes a new closed-loop paradigm for synthesizing safe control policies that explicitly account for the robot's evolving uncertainty.
  arXiv  Detail & Related papers  (2023-09-03T20:34:01Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learning [0.0]
  This work adopts control barrier functions over reinforcement learning, and proposes a compensated algorithm to completely maintain safety. Compared to quadratic programming based reinforcement learning methods, our sum-of-squares programming based reinforcement learning has shown its superiority.
  arXiv  Detail & Related papers  (2022-06-16T04:38:50Z)
• Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning [121.9708998627352]
  Recent work has shown that, in practical robot learning applications, the effects of adversarial training do not pose a fair trade-off. This work revisits the robustness-accuracy trade-off in robot learning by analyzing if recent advances in robust training methods and theory can make adversarial training suitable for real-world robot applications.
  arXiv  Detail & Related papers  (2022-04-15T08:12:15Z)
• safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning [3.9258421820410225]
  We propose a new open-source benchmark suite, called safe-control-gym. Our starting point is OpenAI's Gym API, which is one of the de facto standard in reinforcement learning research. We show how to use safe-control-gym to quantitatively compare the control performance, data efficiency, and safety of multiple approaches.
  arXiv  Detail & Related papers  (2021-09-13T21:09:28Z)
• Learning to be Safe: Deep RL with a Safety Critic [72.00568333130391]
  A natural first approach toward safe RL is to manually specify constraints on the policy's behavior. We propose to learn how to be safe in one set of tasks and environments, and then use that learned intuition to constrain future behaviors.
  arXiv  Detail & Related papers  (2020-10-27T20:53:20Z)
• Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies [47.08581439933752]
  We propose an automated black box testing framework based on adversarial reinforcement learning. We show that the proposed framework is able to find weaknesses in both control policies that were not evident during online testing.
  arXiv  Detail & Related papers  (2020-02-27T13:14:53Z)
• Cautious Reinforcement Learning with Logical Constraints [78.96597639789279]
  An adaptive safe padding forces Reinforcement Learning (RL) to synthesise optimal control policies while ensuring safety during the learning process. Theoretical guarantees are available on the optimality of the synthesised policies and on the convergence of the learning algorithm.
  arXiv  Detail & Related papers  (2020-02-26T00:01:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.