Chance-Constrained Control with Lexicographic Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2010.09468v1
• Date: Mon, 19 Oct 2020 13:09:14 GMT
• Title: Chance-Constrained Control with Lexicographic Deep Reinforcement Learning
• Authors: Alessandro Giuseppi, Antonio Pietrabissa
• Abstract summary: This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.
• Score: 77.34726150561087
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes, in which the controller seeks to ensure that the probability of satisfying the constraint is above a given threshold. Standard DeepRL approaches require i) the constraints to be included as additional weighted terms in the cost function, in a multi-objective fashion, and ii) the tuning of the introduced weights during the training phase of the Deep Neural Network (DNN) according to the probability thresholds. The proposed approach, instead, requires to separately train one constraint-free DNN and one DNN associated to each constraint and then, at each time-step, to select which DNN to use depending on the system observed state. The presented solution does not require any hyper-parameter tuning besides the standard DNN ones, even if the probability thresholds changes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.

Related papers

• Achieving Constraints in Neural Networks: A Stochastic Augmented Lagrangian Approach [49.1574468325115]
  Regularizing Deep Neural Networks (DNNs) is essential for improving generalizability and preventing overfitting. We propose a novel approach to DNN regularization by framing the training process as a constrained optimization problem. We employ the Augmented Lagrangian (SAL) method to achieve a more flexible and efficient regularization mechanism.
  arXiv  Detail & Related papers  (2023-10-25T13:55:35Z)
• Probabilistic Reach-Avoid for Bayesian Neural Networks [71.67052234622781]
  We show that an optimal synthesis algorithm can provide more than a four-fold increase in the number of certifiable states. The algorithm is able to provide more than a three-fold increase in the average guaranteed reach-avoid probability.
  arXiv  Detail & Related papers  (2023-10-03T10:52:21Z)
• Constrained Empirical Risk Minimization: Theory and Practice [2.4934936799100034]
  We present a framework that allows the exact enforcement of constraints on parameterized sets of functions such as Deep Neural Networks (DNNs) We focus on constraints that are outside the scope of equivariant networks used in Geometric Deep Learning. As a major example of the framework, we restrict filters of a Convolutional Neural Network (CNN) to be wavelets, and apply these wavelet networks to the task of contour prediction in the medical domain.
  arXiv  Detail & Related papers  (2023-02-09T16:11:58Z)
• The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networks [94.63547069706459]
  #DNN-Verification problem involves counting the number of input configurations of a DNN that result in a violation of a safety property. We propose a novel approach that returns the exact count of violations. We present experimental results on a set of safety-critical benchmarks.
  arXiv  Detail & Related papers  (2023-01-17T18:32:01Z)
• POLICE: Provably Optimal Linear Constraint Enforcement for Deep Neural Networks [19.587273175563745]
  We propose the first provable affine constraint enforcement method for Deep Neural Networks (DNNs) Our method does not require any sampling and provably ensures that the DNN fulfills the affine constraint on a given input space's region at any point during training, and testing.
  arXiv  Detail & Related papers  (2022-11-02T17:48:52Z)
• Certification of Iterative Predictions in Bayesian Neural Networks [79.15007746660211]
  We compute lower bounds for the probability that trajectories of the BNN model reach a given set of states while avoiding a set of unsafe states. We use the lower bounds in the context of control and reinforcement learning to provide safety certification for given control policies.
  arXiv  Detail & Related papers  (2021-05-21T05:23:57Z)
• On the Turnpike to Design of Deep Neural Nets: Explicit Depth Bounds [0.0]
  This paper attempts a quantifiable answer to the question of how many layers should be considered in a Deep Neural Networks (DNN) The underlying assumption is that the number of neurons per layer -- i.e., the width of the DNN -- is kept constant. We prove explicit bounds on the required depths of DNNs based on reachability of assumptions and a dissipativity-inducing choice of the regularization terms in the training problem.
  arXiv  Detail & Related papers  (2021-01-08T13:23:37Z)
• Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channels [69.18343801164741]
  partitioned edge learning (PARTEL) implements parameter-server training, a well known distributed learning method, in wireless network. We consider the case of deep neural network (DNN) models which can be trained using PARTEL by introducing some auxiliary variables.
  arXiv  Detail & Related papers  (2020-10-08T15:27:50Z)
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness [24.473250414880454]
  We study principled approaches to high-quality uncertainty estimation that require only a single deep neural network (DNN) By formalizing the uncertainty quantification as a minimax learning problem, we first identify input distance awareness, i.e., the model's ability to quantify the distance of a testing example from the training data in the input space. We then propose Spectral-normalized Neural Gaussian Process (SNGP), a simple method that improves the distance-awareness ability of modern DNNs.
  arXiv  Detail & Related papers  (2020-06-17T19:18:22Z)
• Verifiable RNN-Based Policies for POMDPs Under Temporal Logic Constraints [31.829932777445894]
  A major drawback in the application of RNN-based policies is the difficulty in providing formal guarantees on the satisfaction of behavioral specifications. By integrating techniques from formal methods and machine learning, we propose an approach to automatically extract a finite-state controller from an RNN.
  arXiv  Detail & Related papers  (2020-02-13T16:38:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.