CDT: Cascading Decision Trees for Explainable Reinforcement Learning
- URL: http://arxiv.org/abs/2011.07553v2
- Date: Tue, 30 Mar 2021 10:40:38 GMT
- Title: CDT: Cascading Decision Trees for Explainable Reinforcement Learning
- Authors: Zihan Ding, Pablo Hernandez-Leal, Gavin Weiguang Ding, Changjian Li,
Ruitong Huang
- Abstract summary: Cascading Decision Trees (CDTs) apply representation learning on the decision path to allow richer expressivity.
As a second contribution our study reveals limitations of explaining black-box policies via imitation learning with tree-based explainable models, due to its inherent instability.
- Score: 19.363238773001537
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep Reinforcement Learning (DRL) has recently achieved significant advances
in various domains. However, explaining the policy of RL agents still remains
an open problem due to several factors, one being the complexity of explaining
neural networks decisions. Recently, a group of works have used
decision-tree-based models to learn explainable policies. Soft decision trees
(SDTs) and discretized differentiable decision trees (DDTs) have been
demonstrated to achieve both good performance and share the benefit of having
explainable policies. In this work, we further improve the results for
tree-based explainable RL in both performance and explainability. Our proposal,
Cascading Decision Trees (CDTs) apply representation learning on the decision
path to allow richer expressivity. Empirical results show that in both
situations, where CDTs are used as policy function approximators or as
imitation learners to explain black-box policies, CDTs can achieve better
performances with more succinct and explainable models than SDTs. As a second
contribution our study reveals limitations of explaining black-box policies via
imitation learning with tree-based explainable models, due to its inherent
instability.
Related papers
- Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling [3.890533943135602]
We present a framework called IRL (Interpretable Reinforcement Learning) to address the issue of interpretability of DRL scheduling.
ILR is capable of converting a black-box DNN policy into an interpretable rulebased decision tree while maintaining comparable scheduling performance.
arXiv Detail & Related papers (2024-03-24T20:56:16Z) - Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression [53.33734159983431]
This paper introduces a novel approach to distill neural RL policies into more interpretable forms.
We train expert neural network policies using RL and distill them into (i) GBMs, (ii) EBMs, and (iii) symbolic policies.
arXiv Detail & Related papers (2024-03-21T11:54:45Z) - Causal State Distillation for Explainable Reinforcement Learning [16.998047658978482]
Reinforcement learning (RL) is a powerful technique for training intelligent agents, but understanding why these agents make specific decisions can be challenging.
Various approaches have been explored to address this problem, with one promising avenue being reward decomposition (RD)
RD is appealing as it sidesteps some of the concerns associated with other methods that attempt to rationalize an agent's behaviour in a post-hoc manner.
We present an extension of RD that goes beyond sub-rewards to provide more informative explanations.
arXiv Detail & Related papers (2023-12-30T00:01:22Z) - Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
We propose to expose captured knowledge in the form of a directed acyclic causal graph.
We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs.
The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
arXiv Detail & Related papers (2023-09-30T20:59:42Z) - Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in
IBMDPs [9.587070290189507]
Interpretability of AI models allows for user safety checks to build trust in such AIs.
Decision Trees (DTs) provide a global look at the learned model and transparently reveal which features of the input are critical for making a decision.
Recent Reinforcement Learning framework has been proposed to explore the space of DTs using deep RL.
arXiv Detail & Related papers (2023-09-23T13:06:20Z) - Interpretable and Explainable Logical Policies via Neurally Guided
Symbolic Abstraction [23.552659248243806]
We introduce Neurally gUided Differentiable loGic policiEs (NUDGE)
NUDGE exploits trained neural network-based agents to guide the search of candidate-weighted logic rules, then uses differentiable logic to train the logic agents.
Our experimental evaluation demonstrates that NUDGE agents can induce interpretable and explainable policies while outperforming purely neural ones and showing good flexibility to environments of different initial states and problem sizes.
arXiv Detail & Related papers (2023-06-02T10:59:44Z) - Complementary Explanations for Effective In-Context Learning [77.83124315634386]
Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts.
This work aims to better understand the mechanisms by which explanations are used for in-context learning.
arXiv Detail & Related papers (2022-11-25T04:40:47Z) - On Tackling Explanation Redundancy in Decision Trees [19.833126971063724]
Decision trees (DTs) epitomize the ideal of interpretability of machine learning (ML) models.
This paper offers both theoretical and experimental arguments demonstrating that, as long as interpretability of decision trees equates with succinctness of explanations, then decision trees ought not to be deemed interpretable.
arXiv Detail & Related papers (2022-05-20T05:33:38Z) - Collective eXplainable AI: Explaining Cooperative Strategies and Agent
Contribution in Multiagent Reinforcement Learning with Shapley Values [68.8204255655161]
This study proposes a novel approach to explain cooperative strategies in multiagent RL using Shapley values.
Results could have implications for non-discriminatory decision making, ethical and responsible AI-derived decisions or policy making under fairness constraints.
arXiv Detail & Related papers (2021-10-04T10:28:57Z) - Rectified Decision Trees: Exploring the Landscape of Interpretable and
Effective Machine Learning [66.01622034708319]
We propose a knowledge distillation based decision trees extension, dubbed rectified decision trees (ReDT)
We extend the splitting criteria and the ending condition of the standard decision trees, which allows training with soft labels.
We then train the ReDT based on the soft label distilled from a well-trained teacher model through a novel jackknife-based method.
arXiv Detail & Related papers (2020-08-21T10:45:25Z) - Explainability in Deep Reinforcement Learning [68.8204255655161]
We review recent works in the direction to attain Explainable Reinforcement Learning (XRL)
In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
arXiv Detail & Related papers (2020-08-15T10:11:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.