Provably Safe Model-Based Meta Reinforcement Learning: An Abstraction-Based Approach

• URL: http://arxiv.org/abs/2109.01255v1
• Date: Fri, 3 Sep 2021 00:38:05 GMT
• Title: Provably Safe Model-Based Meta Reinforcement Learning: An Abstraction-Based Approach
• Authors: Xiaowu Sun, Wael Fatnassi, Ulices Santa Cruz, and Yasser Shoukry
• Abstract summary: We consider the problem of training a provably safe Neural Network (NN) controller for uncertain nonlinear dynamical systems. Our approach is to learn a set of NN controllers during the training phase. When the task becomes available at runtime, our framework will carefully select a subset of these NN controllers and compose them to form the final NN controller.
• Score: 3.569867801312134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While conventional reinforcement learning focuses on designing agents that can perform one task, meta-learning aims, instead, to solve the problem of designing agents that can generalize to different tasks (e.g., environments, obstacles, and goals) that were not considered during the design or the training of these agents. In this spirit, in this paper, we consider the problem of training a provably safe Neural Network (NN) controller for uncertain nonlinear dynamical systems that can generalize to new tasks that were not present in the training data while preserving strong safety guarantees. Our approach is to learn a set of NN controllers during the training phase. When the task becomes available at runtime, our framework will carefully select a subset of these NN controllers and compose them to form the final NN controller. Critical to our approach is the ability to compute a finite-state abstraction of the nonlinear dynamical system. This abstract model captures the behavior of the closed-loop system under all possible NN weights, and is used to train the NNs and compose them when the task becomes available. We provide theoretical guarantees that govern the correctness of the resulting NN. We evaluated our approach on the problem of controlling a wheeled robot in cluttered environments that were not present in the training data.

Related papers

• Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch [72.26822499434446]
  Auto-Train-Once (ATO) is an innovative network pruning algorithm designed to automatically reduce the computational and storage costs of DNNs. We provide a comprehensive convergence analysis as well as extensive experiments, and the results show that our approach achieves state-of-the-art performance across various model architectures.
  arXiv  Detail & Related papers  (2024-03-21T02:33:37Z)
• Verified Compositional Neuro-Symbolic Control for Stochastic Systems with Temporal Logic Tasks [11.614036749291216]
  Several methods have been proposed recently to learn neural network (NN) controllers for autonomous agents. A key challenge within these approaches is that they often lack safety guarantees or the provided guarantees are impractical. This paper aims to address this challenge by checking if there exists a temporal composition of the trained NN controllers.
  arXiv  Detail & Related papers  (2023-11-17T20:51:24Z)
• Safety Filter Design for Neural Network Systems via Convex Optimization [35.87465363928146]
  We propose a novel safety filter that relies on convex optimization to ensure safety for a neural network (NN) system. We demonstrate the efficacy of the proposed framework numerically on a nonlinear pendulum system.
  arXiv  Detail & Related papers  (2023-08-16T01:30:13Z)
• ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction [82.81767856234956]
  This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling. We show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics.
  arXiv  Detail & Related papers  (2023-02-11T21:07:30Z)
• Quantization-aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks [58.195261590442406]
  We study the problem of training and certifying adversarially robust quantized neural networks (QNNs) Recent work has shown that floating-point neural networks that have been verified to be robust can become vulnerable to adversarial attacks after quantization. We present quantization-aware interval bound propagation (QA-IBP), a novel method for training robust QNNs.
  arXiv  Detail & Related papers  (2022-11-29T13:32:38Z)
• Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systems [59.57462129637796]
  This paper presents a backward reachability approach for safety verification of closed-loop systems with neural networks (NNs) The presence of NNs in the feedback loop presents a unique set of problems due to the nonlinearities in their activation functions and because NN models are generally not invertible. We present frameworks for calculating BP over-approximations for both linear and nonlinear systems with control policies represented by feedforward NNs.
  arXiv  Detail & Related papers  (2022-09-28T13:17:28Z)
• Learning To Estimate Regions Of Attraction Of Autonomous Dynamical Systems Using Physics-Informed Neural Networks [0.0]
  We train a neural network to estimate the region of attraction (ROA) of a controlled autonomous dynamical system. This safety network can be used to quantify the relative safety of proposed control actions and prevent the selection of damaging actions. In future work we intend to apply this technique to reinforcement learning agents during motor learning tasks.
  arXiv  Detail & Related papers  (2021-11-18T19:58:47Z)
• Provably Correct Training of Neural Network Controllers Using Reachability Analysis [3.04585143845864]
  We consider the problem of training neural network (NN) controllers for cyber-physical systems that are guaranteed to satisfy safety and liveness properties. Our approach is to combine model-based design methodologies for dynamical systems with data-driven approaches to achieve this target.
  arXiv  Detail & Related papers  (2021-02-22T07:08:11Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• PaRoT: A Practical Framework for Robust Deep Neural Network Training [1.9034855801255839]
  Deep Neural Networks (DNNs) are finding important applications in safety-critical systems such as Autonomous Vehicles (AVs) Raising unique challenges for assurance due to their black-box nature, DNNs pose a fundamental problem for regulatory acceptance of these types of systems. We introduce a novel framework, PaRoT, developed on a popular training platform, that greatly reduces the barrier to entry.
  arXiv  Detail & Related papers  (2020-01-07T16:21:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.