Deep Binary Reinforcement Learning for Scalable Verification

• URL: http://arxiv.org/abs/2203.05704v1
• Date: Fri, 11 Mar 2022 01:20:23 GMT
• Title: Deep Binary Reinforcement Learning for Scalable Verification
• Authors: Christopher Lazarus and Mykel J. Kochenderfer
• Abstract summary: We present an RL algorithm tailored specifically for binarized neural networks (BNNs) After training BNNs for the Atari environments, we verify robustness properties.
• Score: 44.44006029119672
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of neural networks as function approximators has enabled many advances in reinforcement learning (RL). The generalization power of neural networks combined with advances in RL algorithms has reignited the field of artificial intelligence. Despite their power, neural networks are considered black boxes, and their use in safety-critical settings remains a challenge. Recently, neural network verification has emerged as a way to certify safety properties of networks. Verification is a hard problem, and it is difficult to scale to large networks such as the ones used in deep reinforcement learning. We provide an approach to train RL policies that are more easily verifiable. We use binarized neural networks (BNNs), a type of network with mostly binary parameters. We present an RL algorithm tailored specifically for BNNs. After training BNNs for the Atari environments, we verify robustness properties.

Related papers

• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• Adversarial Training Using Feedback Loops [1.6114012813668932]
  Deep neural networks (DNNs) are highly susceptible to adversarial attacks due to limited generalizability. This paper proposes a new robustification approach based on control theory. The novel adversarial training approach based on the feedback control architecture is called Feedback Looped Adversarial Training (FLAT)
  arXiv  Detail & Related papers  (2023-08-23T02:58:02Z)
• Uncertainty Quantification and Resource-Demanding Computer Vision Applications of Deep Learning [5.130440339897478]
  Bringing deep neural networks (DNNs) into safety critical applications requires a thorough treatment of the model's uncertainties. In this article, we survey methods that we developed to teach DNNs to be uncertain when they encounter new object classes. We also present training methods to learn from only a few labels with help of uncertainty quantification.
  arXiv  Detail & Related papers  (2022-05-30T08:31:03Z)
• HyBNN and FedHyBNN: (Federated) Hybrid Binary Neural Networks [0.0]
  We introduce a novel hybrid neural network architecture, Hybrid Binary Neural Network (HyBNN) HyBNN consists of a task-independent, general, full-precision variational autoencoder with a binary latent space and a task specific binary neural network. We show that our proposed system is able to very significantly outperform a vanilla binary neural network with input binarization.
  arXiv  Detail & Related papers  (2022-05-19T20:27:01Z)
• Deep Reinforcement Learning with Spiking Q-learning [51.386945803485084]
  spiking neural networks (SNNs) are expected to realize artificial intelligence (AI) with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combining SNNs with deep reinforcement learning (RL)
  arXiv  Detail & Related papers  (2022-01-21T16:42:11Z)
• Provable Regret Bounds for Deep Online Learning and Control [77.77295247296041]
  We show that any loss functions can be adapted to optimize the parameters of a neural network such that it competes with the best net in hindsight. As an application of these results in the online setting, we obtain provable bounds for online control controllers.
  arXiv  Detail & Related papers  (2021-10-15T02:13:48Z)
• Mining the Weights Knowledge for Optimizing Neural Network Structures [1.995792341399967]
  We introduce a switcher neural network (SNN) that uses as inputs the weights of a task-specific neural network (called TNN for short) By mining the knowledge contained in the weights, the SNN outputs scaling factors for turning off neurons in the TNN. In terms of accuracy, we outperform baseline networks and other structure learning methods stably and significantly.
  arXiv  Detail & Related papers  (2021-10-11T05:20:56Z)
• Building Compact and Robust Deep Neural Networks with Toeplitz Matrices [93.05076144491146]
  This thesis focuses on the problem of training neural networks which are compact, easy to train, reliable and robust to adversarial examples. We leverage the properties of structured matrices from the Toeplitz family to build compact and secure neural networks.
  arXiv  Detail & Related papers  (2021-09-02T13:58:12Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.