Explainable Reinforcement Learning: A Survey
- URL: http://arxiv.org/abs/2005.06247v1
- Date: Wed, 13 May 2020 10:52:49 GMT
- Title: Explainable Reinforcement Learning: A Survey
- Authors: Erika Puiutta and Eric MSP Veith
- Abstract summary: Explainable Artificial Intelligence (XAI) has gained increased traction over the last few years.
XAI models exhibit one detrimential characteristic: a performance-transparency trade-off.
This survey attempts to address this gap by offering an overview of Explainable Reinforcement Learning (XRL) methods.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Explainable Artificial Intelligence (XAI), i.e., the development of more
transparent and interpretable AI models, has gained increased traction over the
last few years. This is due to the fact that, in conjunction with their growth
into powerful and ubiquitous tools, AI models exhibit one detrimential
characteristic: a performance-transparency trade-off. This describes the fact
that the more complex a model's inner workings, the less clear it is how its
predictions or decisions were achieved. But, especially considering Machine
Learning (ML) methods like Reinforcement Learning (RL) where the system learns
autonomously, the necessity to understand the underlying reasoning for their
decisions becomes apparent. Since, to the best of our knowledge, there exists
no single work offering an overview of Explainable Reinforcement Learning (XRL)
methods, this survey attempts to address this gap. We give a short summary of
the problem, a definition of important terms, and offer a classification and
assessment of current XRL methods. We found that a) the majority of XRL methods
function by mimicking and simplifying a complex model instead of designing an
inherently simple one, and b) XRL (and XAI) methods often neglect to consider
the human side of the equation, not taking into account research from related
fields like psychology or philosophy. Thus, an interdisciplinary effort is
needed to adapt the generated explanations to a (non-expert) human user in
order to effectively progress in the field of XRL and XAI in general.
Related papers
- Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment [10.814585613336778]
Causal representation learning aims to combine the core strengths of machine learning and causality.
This thesis investigates what is possible for CRL without direct supervision, and thus contributes to its theoretical foundations.
arXiv Detail & Related papers (2024-06-19T09:14:40Z) - Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning [50.47568731994238]
Key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL)
This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies.
arXiv Detail & Related papers (2023-12-22T17:57:57Z) - A Survey on Explainable Reinforcement Learning: Concepts, Algorithms,
Challenges [38.70863329476517]
Reinforcement Learning (RL) is a popular machine learning paradigm where intelligent agents interact with the environment to fulfill a long-term goal.
Despite the encouraging results achieved, the deep neural network-based backbone is widely deemed as a black box that impedes practitioners to trust and employ trained agents in realistic scenarios where high security and reliability are essential.
To alleviate this issue, a large volume of literature devoted to shedding light on the inner workings of the intelligent agents has been proposed, by constructing intrinsic interpretability or post-hoc explainability.
arXiv Detail & Related papers (2022-11-12T13:52:06Z) - Explainability in reinforcement learning: perspective and position [1.299941371793082]
This paper attempts to give a systematic overview of existing methods in the explainable RL area.
It proposes a novel unified taxonomy, building and expanding on the existing ones.
arXiv Detail & Related papers (2022-03-22T09:00:13Z) - RvS: What is Essential for Offline RL via Supervised Learning? [77.91045677562802]
Recent work has shown that supervised learning alone, without temporal difference (TD) learning, can be remarkably effective for offline RL.
In every environment suite we consider simply maximizing likelihood with two-layer feedforward is competitive.
They also probe the limits of existing RvS methods, which are comparatively weak on random data.
arXiv Detail & Related papers (2021-12-20T18:55:16Z) - Exploratory State Representation Learning [63.942632088208505]
We propose a new approach called XSRL (eXploratory State Representation Learning) to solve the problems of exploration and SRL in parallel.
On one hand, it jointly learns compact state representations and a state transition estimator which is used to remove unexploitable information from the representations.
On the other hand, it continuously trains an inverse model, and adds to the prediction error of this model a $k$-step learning progress bonus to form the objective of a discovery policy.
arXiv Detail & Related papers (2021-09-28T10:11:07Z) - Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework
and Survey [0.7366405857677226]
Reinforcement Learning (RL) methods provide a potential backbone for the cognitive model required for the development of Broad-XAI.
RL represents a suite of approaches that have had increasing success in solving a range of sequential decision-making problems.
This paper aims to introduce a conceptual framework, called the Causal XRL Framework (CXF), that unifies the current XRL research and uses RL as a backbone to the development of Broad-XAI.
arXiv Detail & Related papers (2021-08-20T05:18:50Z) - Explainability in Deep Reinforcement Learning [68.8204255655161]
We review recent works in the direction to attain Explainable Reinforcement Learning (XRL)
In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
arXiv Detail & Related papers (2020-08-15T10:11:42Z) - Efficient Model-Based Reinforcement Learning through Optimistic Policy
Search and Planning [93.1435980666675]
We show how optimistic exploration can be easily combined with state-of-the-art reinforcement learning algorithms.
Our experiments demonstrate that optimistic exploration significantly speeds-up learning when there are penalties on actions.
arXiv Detail & Related papers (2020-06-15T18:37:38Z) - Explainable Active Learning (XAL): An Empirical Study of How Local
Explanations Impact Annotator Experience [76.9910678786031]
We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting.
Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
arXiv Detail & Related papers (2020-01-24T22:52:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.