Towards Interpretable Deep Reinforcement Learning Models via Inverse
Reinforcement Learning
- URL: http://arxiv.org/abs/2203.16464v3
- Date: Fri, 1 Mar 2024 18:40:56 GMT
- Title: Towards Interpretable Deep Reinforcement Learning Models via Inverse
Reinforcement Learning
- Authors: Sean Xie, Soroush Vosoughi, Saeed Hassanpour
- Abstract summary: We propose a novel framework utilizing Adversarial Inverse Reinforcement Learning.
This framework provides global explanations for decisions made by a Reinforcement Learning model.
We capture intuitive tendencies that the model follows by summarizing the model's decision-making process.
- Score: 27.841725567976315
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Artificial intelligence, particularly through recent advancements in deep
learning, has achieved exceptional performances in many tasks in fields such as
natural language processing and computer vision. In addition to desirable
evaluation metrics, a high level of interpretability is often required for
these models to be reliably utilized. Therefore, explanations that offer
insight into the process by which a model maps its inputs onto its outputs are
much sought-after. Unfortunately, the current black box nature of machine
learning models is still an unresolved issue and this very nature prevents
researchers from learning and providing explicative descriptions for a model's
behavior and final predictions. In this work, we propose a novel framework
utilizing Adversarial Inverse Reinforcement Learning that can provide global
explanations for decisions made by a Reinforcement Learning model and capture
intuitive tendencies that the model follows by summarizing the model's
decision-making process.
Related papers
- Deep Learning for Robust and Explainable Models in Computer Vision [0.0]
This thesis presents various approaches that address robustness and explainability challenges for using ML and DL in practice.
This thesis presents developments in computer vision models' robustness and explainability.
In addition to the theoretical developments, this thesis demonstrates several applications of ML and DL in different contexts.
arXiv Detail & Related papers (2024-03-27T15:17:10Z) - Model-Agnostic Interpretation Framework in Machine Learning: A
Comparative Study in NBA Sports [0.2937071029942259]
We propose an innovative framework to reconcile the trade-off between model performance and interpretability.
Our approach is centered around modular operations on high-dimensional data, which enable end-to-end processing while preserving interpretability.
We have extensively tested our framework and validated its superior efficacy in achieving a balance between computational efficiency and interpretability.
arXiv Detail & Related papers (2024-01-05T04:25:21Z) - Explainability for Large Language Models: A Survey [59.67574757137078]
Large language models (LLMs) have demonstrated impressive capabilities in natural language processing.
This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
arXiv Detail & Related papers (2023-09-02T22:14:26Z) - Beyond Explaining: Opportunities and Challenges of XAI-Based Model
Improvement [75.00655434905417]
Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models.
This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models.
We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
arXiv Detail & Related papers (2022-03-15T15:44:28Z) - Analyzing a Caching Model [7.378507865227209]
Interpretability remains a major obstacle for adoption in real-world deployments.
By analyzing a state-of-the-art caching model, we provide evidence that the model has learned concepts beyond simple statistics.
arXiv Detail & Related papers (2021-12-13T19:53:07Z) - Beyond Trivial Counterfactual Explanations with Diverse Valuable
Explanations [64.85696493596821]
In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction.
We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss.
Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
arXiv Detail & Related papers (2021-03-18T12:57:34Z) - Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy.
We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space.
We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
arXiv Detail & Related papers (2020-07-14T16:42:59Z) - Towards Interpretable Deep Learning Models for Knowledge Tracing [62.75876617721375]
We propose to adopt the post-hoc method to tackle the interpretability issue for deep learning based knowledge tracing (DLKT) models.
Specifically, we focus on applying the layer-wise relevance propagation (LRP) method to interpret RNN-based DLKT model.
Experiment results show the feasibility using the LRP method for interpreting the DLKT model's predictions.
arXiv Detail & Related papers (2020-05-13T04:03:21Z) - Plausible Counterfactuals: Auditing Deep Learning Classifiers with
Realistic Adversarial Examples [84.8370546614042]
Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data.
Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model.
Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
arXiv Detail & Related papers (2020-03-25T11:08:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.