Modeling Risk in Reinforcement Learning: A Literature Mapping
- URL: http://arxiv.org/abs/2312.05231v1
- Date: Fri, 8 Dec 2023 18:26:08 GMT
- Title: Modeling Risk in Reinforcement Learning: A Literature Mapping
- Authors: Leonardo Villalobos-Arias, Derek Martin, Abhijeet Krishnan, Madeleine
Gagn\'e, Colin M. Potts, Arnav Jhala
- Abstract summary: We perform a systematic literature mapping with the objective to characterize risk in safe RL.
Based on the obtained results, we present definitions, characteristics, and types of risk that hold on multiple application domains.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Safe reinforcement learning deals with mitigating or avoiding unsafe
situations by reinforcement learning (RL) agents. Safe RL approaches are based
on specific risk representations for particular problems or domains. In order
to analyze agent behaviors, compare safe RL approaches, and effectively
transfer techniques between application domains, it is necessary to understand
the types of risk specific to safe RL problems. We performed a systematic
literature mapping with the objective to characterize risk in safe RL. Based on
the obtained results, we present definitions, characteristics, and types of
risk that hold on multiple application domains. Our literature mapping covers
literature from the last 5 years (2017-2022), from a variety of knowledge areas
(AI, finance, engineering, medicine) where RL approaches emphasize risk
representation and management. Our mapping covers 72 papers filtered
systematically from over thousands of papers on the topic. Our proposed notion
of risk covers a variety of representations, disciplinary differences, common
training exercises, and types of techniques. We encourage researchers to
include explicit and detailed accounts of risk in future safe RL research
reports, using this mapping as a starting point. With this information,
researchers and practitioners could draw stronger conclusions on the
effectiveness of techniques on different problems.
Related papers
- Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents [67.07177243654485]
This survey collects and analyzes the different threats faced by large language models-based agents.
We identify six key features of LLM-based agents, based on which we summarize the current research progress.
We select four representative agents as case studies to analyze the risks they may face in practical use.
arXiv Detail & Related papers (2024-11-14T15:40:04Z) - Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users.
We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
arXiv Detail & Related papers (2024-08-16T17:23:43Z) - Safe and Robust Reinforcement Learning: Principles and Practice [0.0]
Reinforcement Learning has shown remarkable success in solving relatively complex tasks.
The deployment of RL systems in real-world scenarios poses significant challenges related to safety and robustness.
This paper explores the main dimensions of the safe and robust RL landscape, encompassing algorithmic, ethical, and practical considerations.
arXiv Detail & Related papers (2024-03-27T13:14:29Z) - Provable Risk-Sensitive Distributional Reinforcement Learning with
General Function Approximation [54.61816424792866]
We introduce a general framework on Risk-Sensitive Distributional Reinforcement Learning (RS-DisRL), with static Lipschitz Risk Measures (LRM) and general function approximation.
We design two innovative meta-algorithms: textttRS-DisRL-M, a model-based strategy for model-based function approximation, and textttRS-DisRL-V, a model-free approach for general value function approximation.
arXiv Detail & Related papers (2024-02-28T08:43:18Z) - Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification.
We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations.
Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
arXiv Detail & Related papers (2024-02-07T21:58:40Z) - Safeguarded Progress in Reinforcement Learning: Safe Bayesian
Exploration for Control Policy Synthesis [63.532413807686524]
This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL)
We propose a new architecture that handles the trade-off between efficient progress and safety during exploration.
arXiv Detail & Related papers (2023-12-18T16:09:43Z) - Is Risk-Sensitive Reinforcement Learning Properly Resolved? [32.42976780682353]
We propose a novel algorithm, namely Trajectory Q-Learning (TQL), for RSRL problems with provable convergence to the optimal policy.
Based on our new learning architecture, we are free to introduce a general and practical implementation for different risk measures to learn disparate risk-sensitive policies.
arXiv Detail & Related papers (2023-07-02T11:47:21Z) - TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI [11.240642213359267]
Many exhaustive taxonomy are possible, and some are useful -- particularly if they reveal new risks or practical approaches to safety.
This paper explores a taxonomy based on accountability: whose actions lead to the risk, are the actors unified, and are they deliberate?
We also provide stories to illustrate how the various risk types could each play out, including risks arising from unanticipated interactions of many AI systems, and risks from deliberate misuse.
arXiv Detail & Related papers (2023-06-12T07:55:18Z) - UQ for Credit Risk Management: A deep evidence regression approach [3.42658286826597]
We have explored the application of a scalable UQ-aware deep learning technique, Deep Evidence Regression and applied it to predicting Loss Given Default.
We contribute to the literature by extending the Deep Evidence Regression methodology to learning target variables generated by a Weibull process.
We demonstrate the application of our approach to both simulated and real-world data.
arXiv Detail & Related papers (2023-05-08T18:03:01Z) - A Survey of Risk-Aware Multi-Armed Bandits [84.67376599822569]
We review various risk measures of interest, and comment on their properties.
We consider algorithms for the regret minimization setting, where the exploration-exploitation trade-off manifests.
We conclude by commenting on persisting challenges and fertile areas for future research.
arXiv Detail & Related papers (2022-05-12T02:20:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.