Evaluating the Safety of Deep Reinforcement Learning Models using Semi-Formal Verification

• URL: http://arxiv.org/abs/2010.09387v1
• Date: Mon, 19 Oct 2020 11:18:06 GMT
• Title: Evaluating the Safety of Deep Reinforcement Learning Models using Semi-Formal Verification
• Authors: Davide Corsi, Enrico Marchesini, Alessandro Farinelli
• Abstract summary: We present a semi-formal verification approach for decision-making tasks based on interval analysis. Our method obtains comparable results over standard benchmarks with respect to formal verifiers. Our approach allows to efficiently evaluate safety properties for decision-making models in practical applications.
• Score: 81.32981236437395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Groundbreaking successes have been achieved by Deep Reinforcement Learning (DRL) in solving practical decision-making problems. Robotics, in particular, can involve high-cost hardware and human interactions. Hence, scrupulous evaluations of trained models are required to avoid unsafe behaviours in the operational environment. However, designing metrics to measure the safety of a neural network is an open problem, since standard evaluation parameters (e.g., total reward) are not informative enough. In this paper, we present a semi-formal verification approach for decision-making tasks, based on interval analysis, that addresses the computational demanding of previous verification frameworks and design metrics to measure the safety of the models. Our method obtains comparable results over standard benchmarks with respect to formal verifiers, while drastically reducing the computation time. Moreover, our approach allows to efficiently evaluate safety properties for decision-making models in practical applications such as mapless navigation for mobile robots and trajectory generation for manipulators.

Related papers

• Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach [1.0488553716155147]
  This study presents a novel methodology incorporating safety constraints into a robotic simulation during the training of deep reinforcement learning (DRL) The framework integrates specific parts of the safety requirements, such as velocity constraints, directly within the DRL model. The proposed approach outperforms the conventional method by a 16.5% average success rate on the tested scenarios.
  arXiv  Detail & Related papers  (2024-07-02T12:56:17Z)
• Towards Precise Observations of Neural Model Robustness in Classification [2.127049691404299]
  In deep learning applications, robustness measures the ability of neural models that handle slight changes in input data. Our approach contributes to a deeper understanding of model robustness in safety-critical applications.
  arXiv  Detail & Related papers  (2024-04-25T09:37:44Z)
• Towards a Framework for Deep Learning Certification in Safety-Critical Applications Using Inherently Safe Design and Run-Time Error Detection [0.0]
  We consider real-world problems arising in aviation and other safety-critical areas, and investigate their requirements for a certified model. We establish a new framework towards deep learning certification based on (i) inherently safe design, and (ii) run-time error detection.
  arXiv  Detail & Related papers  (2024-03-12T11:38:45Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Online Safety Property Collection and Refinement for Safe Deep Reinforcement Learning in Mapless Navigation [79.89605349842569]
  We introduce the Collection and Refinement of Online Properties (CROP) framework to design properties at training time. CROP employs a cost signal to identify unsafe interactions and use them to shape safety properties. We evaluate our approach in several robotic mapless navigation tasks and demonstrate that the violation metric computed with CROP allows higher returns and lower violations over previous Safe DRL approaches.
  arXiv  Detail & Related papers  (2023-02-13T21:19:36Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation [78.17108227614928]
  We propose a benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation. We consider a value-based and policy-gradient Deep Reinforcement Learning (DRL) We also propose a verification strategy that checks the behavior of the trained models over a set of desired properties.
  arXiv  Detail & Related papers  (2021-12-16T16:53:56Z)
• Reactive and Safe Road User Simulations using Neural Barrier Certificates [9.961324632236499]
  We propose a reactive agent model which can ensure safety without comprising the original purposes. Our learned road user simulation models can achieve a significant improvement in safety. Our learned reactive agents are shown to generalize better to unseen traffic conditions.
  arXiv  Detail & Related papers  (2021-09-14T13:45:37Z)
• Efficient falsification approach for autonomous vehicle validation using a parameter optimisation technique based on reinforcement learning [6.198523595657983]
  The widescale deployment of Autonomous Vehicles (AV) appears to be imminent despite many safety challenges that are yet to be resolved. The uncertainties in the behaviour of the traffic participants and the dynamic world cause reactions in advanced autonomous systems. This paper presents an efficient falsification method to evaluate the System Under Test.
  arXiv  Detail & Related papers  (2020-11-16T02:56:13Z)
• SAMBA: Safe Model-Based & Active Reinforcement Learning [59.01424351231993]
  SAMBA is a framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics. We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations. We provide intuition as to the effectiveness of the framework by a detailed analysis of our active metrics and safety constraints.
  arXiv  Detail & Related papers  (2020-06-12T10:40:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.