Runtime Verification of Learning Properties for Reinforcement Learning Algorithms

• URL: http://arxiv.org/abs/2311.09811v1
• Date: Thu, 16 Nov 2023 11:34:37 GMT
• Title: Runtime Verification of Learning Properties for Reinforcement Learning Algorithms
• Authors: Tommaso Mannucci (TNO -- Netherlands Organisation for Applied Scientific Research), Julio de Oliveira Filho (TNO -- Netherlands Organisation for Applied Scientific Research)
• Abstract summary: Reinforcement learning (RL) algorithms interact with their environment in a trial-and-error fashion. This work develops new runtime verification techniques to predict when the learning phase has not met or will not meet qualitative and timely expectations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) algorithms interact with their environment in a trial-and-error fashion. Such interactions can be expensive, inefficient, and timely when learning on a physical system rather than in a simulation. This work develops new runtime verification techniques to predict when the learning phase has not met or will not meet qualitative and timely expectations. This paper presents three verification properties concerning the quality and timeliness of learning in RL algorithms. With each property, we propose design steps for monitoring and assessing the properties during the system's operation.

Related papers

• A Comparison of Imitation Learning Algorithms for Bimanual Manipulation [22.531439806919547]
  In this work, we demonstrate the limitations and benefits of prominent imitation learning approaches. We evaluate each algorithm on a complex bimanual manipulation task involving an over-constrained dynamics system. We find that imitation learning is well suited to solve such complex tasks, but not all algorithms are equal in terms of handling perturbations, training requirements, performance, and ease of use.
  arXiv  Detail & Related papers  (2024-08-13T00:04:17Z)
• Nature-Inspired Local Propagation [68.63385571967267]
  Natural learning processes rely on mechanisms where data representation and learning are intertwined in such a way as to respect locality. We show that the algorithmic interpretation of the derived "laws of learning", which takes the structure of Hamiltonian equations, reduces to Backpropagation when the speed of propagation goes to infinity. This opens the doors to machine learning based on full on-line information that are based the replacement of Backpropagation with the proposed local algorithm.
  arXiv  Detail & Related papers  (2024-02-04T21:43:37Z)
• A study on a Q-Learning algorithm application to a manufacturing assembly problem [0.8937905773981699]
  This study focuses on the implementation of a reinforcement learning algorithm in an assembly problem of a given object. A model-free Q-Learning algorithm is applied, considering the learning of a matrix of Q-values (Q-table) from the successive interactions with the environment. The optimisation approach achieved very promising results by learning the optimal assembly sequence 98.3% of the times.
  arXiv  Detail & Related papers  (2023-04-17T15:38:34Z)
• Latent Properties of Lifelong Learning Systems [59.50307752165016]
  We introduce an algorithm-agnostic explainable surrogate-modeling approach to estimate latent properties of lifelong learning algorithms. We validate the approach for estimating these properties via experiments on synthetic data.
  arXiv  Detail & Related papers  (2022-07-28T20:58:13Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Accelerated Policy Learning with Parallel Differentiable Simulation [59.665651562534755]
  We present a differentiable simulator and a new policy learning algorithm (SHAC) Our algorithm alleviates problems with local minima through a smooth critic function. We show substantial improvements in sample efficiency and wall-clock time over state-of-the-art RL and differentiable simulation-based algorithms.
  arXiv  Detail & Related papers  (2022-04-14T17:46:26Z)
• Geometric Deep Reinforcement Learning for Dynamic DAG Scheduling [8.14784681248878]
  In this paper, we propose a reinforcement learning approach to solve a realistic scheduling problem. We apply it to an algorithm commonly executed in the high performance computing community, the Cholesky factorization. Our algorithm uses graph neural networks in combination with an actor-critic algorithm (A2C) to build an adaptive representation of the problem on the fly.
  arXiv  Detail & Related papers  (2020-11-09T10:57:21Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)
• Active Learning for Gaussian Process Considering Uncertainties with Application to Shape Control of Composite Fuselage [7.358477502214471]
  We propose two new active learning algorithms for the Gaussian process with uncertainties. We show that the proposed approach can incorporate the impact from uncertainties, and realize better prediction performance. This approach has been applied to improving the predictive modeling for automatic shape control of composite fuselage.
  arXiv  Detail & Related papers  (2020-04-23T02:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.