Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning

• URL: http://arxiv.org/abs/2203.12048v1
• Date: Tue, 22 Mar 2022 21:15:01 GMT
• Title: Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning
• Authors: Laura L. Pullum
• Abstract summary: Reinforcement learning has received significant interest in recent years, due primarily to the successes of deep reinforcement learning. We classify the quantitative and theoretical approaches used to indicate or measure robustness, stability, and resilience behaviors. We believe that this is the first comprehensive review of stability, robustness and resilience specifically geared towards reinforcement learning.
• Score: 0.5917654223291071
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reinforcement learning has received significant interest in recent years, due primarily to the successes of deep reinforcement learning at solving many challenging tasks such as playing Chess, Go and online computer games. However, with the increasing focus on reinforcement learning, applications outside of gaming and simulated environments require understanding the robustness, stability, and resilience of reinforcement learning methods. To this end, we conducted a comprehensive literature review to characterize the available literature on these three behaviors as they pertain to reinforcement learning. We classify the quantitative and theoretical approaches used to indicate or measure robustness, stability, and resilience behaviors. In addition, we identified the action or event to which the quantitative approaches were attempting to be stable, robust, or resilient. Finally, we provide a decision tree useful for selecting metrics to quantify the behaviors. We believe that this is the first comprehensive review of stability, robustness and resilience specifically geared towards reinforcement learning.

Related papers

• A Bag of Tricks for Few-Shot Class-Incremental Learning [20.95422402702963]
  We present a bag of tricks framework for few-shot class-incremental learning (FSCIL) FSCIL requires both stability and adaptability, preserving proficiency in previously learned tasks while learning new ones. We organize these tricks into three categories: stability tricks, adaptability tricks, and training tricks.
  arXiv  Detail & Related papers  (2024-03-21T13:33:00Z)
• Evaluating and Improving Continual Learning in Spoken Language Understanding [58.723320551761525]
  We propose an evaluation methodology that provides a unified evaluation on stability, plasticity, and generalizability in continual learning. By employing the proposed metric, we demonstrate how introducing various knowledge distillations can improve different aspects of these three properties of the SLU model.
  arXiv  Detail & Related papers  (2024-02-16T03:30:27Z)
• Causal Reinforcement Learning: A Survey [57.368108154871]
  Reinforcement learning is an essential paradigm for solving sequential decision problems under uncertainty. One of the main obstacles is that reinforcement learning agents lack a fundamental understanding of the world. Causality offers a notable advantage as it can formalize knowledge in a systematic manner.
  arXiv  Detail & Related papers  (2023-07-04T03:00:43Z)
• Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability [23.82257896376779]
  A trustworthy reinforcement learning algorithm should be competent in solving challenging real-world problems. This study aims to overview these main perspectives of trustworthy reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-16T16:10:08Z)
• A Review of Uncertainty for Deep Reinforcement Learning [0.0]
  Uncertainty is ubiquitous in games, both in the agents playing games and often in the games themselves. This work provides an overview motivating and presenting existing techniques in uncertainty aware deep reinforcement learning.
  arXiv  Detail & Related papers  (2022-08-18T20:42:19Z)
• Continual evaluation for lifelong learning: Identifying the stability gap [35.99653845083381]
  We show that a set of common state-of-the-art methods still suffers from substantial forgetting upon starting to learn new tasks. We refer to this intriguing but potentially problematic phenomenon as the stability gap. We establish a framework for continual evaluation that uses per-iteration evaluation and we define a new set of metrics to quantify worst-case performance.
  arXiv  Detail & Related papers  (2022-05-26T15:56:08Z)
• Rethinking Learning Dynamics in RL using Adversarial Networks [79.56118674435844]
  We present a learning mechanism for reinforcement learning of closely related skills parameterized via a skill embedding space. The main contribution of our work is to formulate an adversarial training regime for reinforcement learning with the help of entropy-regularized policy gradient formulation.
  arXiv  Detail & Related papers  (2022-01-27T19:51:09Z)
• Beyond Robustness: Resilience Verification of Tree-Based Classifiers [7.574509994822738]
  We introduce a new measure called resilience and we focus on its verification. We discuss how resilience can be verified by combining a traditional robustness verification technique with a data-independent stability analysis. Our results show that resilience verification is useful and feasible in practice, yielding a more reliable security assessment of both standard and robust decision tree models.
  arXiv  Detail & Related papers  (2021-12-05T23:07:22Z)
• Transfer Learning in Deep Reinforcement Learning: A Survey [64.36174156782333]
  Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. transfer learning has arisen to tackle various challenges faced by reinforcement learning.
  arXiv  Detail & Related papers  (2020-09-16T18:38:54Z)
• Importance Weighted Policy Learning and Adaptation [89.46467771037054]
  We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning. The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior. Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
  arXiv  Detail & Related papers  (2020-09-10T14:16:58Z)
• Understanding the Role of Training Regimes in Continual Learning [51.32945003239048]
  Catastrophic forgetting affects the training of neural networks, limiting their ability to learn multiple tasks sequentially. We study the effect of dropout, learning rate decay, and batch size, on forming training regimes that widen the tasks' local minima.
  arXiv  Detail & Related papers  (2020-06-12T06:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.