Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System

• URL: http://arxiv.org/abs/2008.01508v2
• Date: Thu, 24 Dec 2020 13:24:37 GMT
• Title: Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System
• Authors: Xinzhi Wang, Huao Li, Hui Zhang, Michael Lewis, Katia Sycara
• Abstract summary: This paper reports a novel work in generating verbal explanations for DRL behaviors agent. A learning model is proposed to expand the implicit logic of generating verbal explanation to general situations. Results show that verbal explanation generated by both models improve subjective satisfaction of users towards the interpretability of DRL systems.
• Score: 3.754171077237216
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, there has been increasing interest in transparency and interpretability in Deep Reinforcement Learning (DRL) systems. Verbal explanations, as the most natural way of communication in our daily life, deserve more attention, since they allow users to gain a better understanding of the system which ultimately could lead to a high level of trust and smooth collaboration. This paper reports a novel work in generating verbal explanations for DRL behaviors agent. A rule-based model is designed to construct explanations using a series of rules which are predefined with prior knowledge. A learning model is then proposed to expand the implicit logic of generating verbal explanation to general situations by employing rule-based explanations as training data. The learning model is shown to have better flexibility and generalizability than the static rule-based model. The performance of both models is evaluated quantitatively through objective metrics. The results show that verbal explanation generated by both models improve subjective satisfaction of users towards the interpretability of DRL systems. Additionally, seven variants of the learning model are designed to illustrate the contribution of input channels, attention mechanism, and proposed encoder in improving the quality of verbal explanation.

Related papers

• Interpret the Internal States of Recommendation Model with Sparse Autoencoder [26.021277330699963]
  RecSAE is an automatic, generalizable probing method for interpreting the internal states of Recommendation models. We train an autoencoder with sparsity constraints to reconstruct internal activations of recommendation models. We automated the construction of concept dictionaries based on the relationship between latent activations and input item sequences.
  arXiv  Detail & Related papers  (2024-11-09T08:22:31Z)
• Semifactual Explanations for Reinforcement Learning [1.5320737596132754]
  Reinforcement Learning (RL) is a learning paradigm in which the agent learns from its environment through trial and error. Deep reinforcement learning (DRL) algorithms represent the agent's policies using neural networks, making their decisions difficult to interpret. Explaining the behaviour of DRL agents is necessary to advance user trust, increase engagement, and facilitate integration with real-life tasks.
  arXiv  Detail & Related papers  (2024-09-09T08:37:47Z)
• Explainability for Machine Learning Models: From Data Adaptability to User Perception [0.8702432681310401]
  This thesis explores the generation of local explanations for already deployed machine learning models. It aims to identify optimal conditions for producing meaningful explanations considering both data and user requirements.
  arXiv  Detail & Related papers  (2024-02-16T18:44:37Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Motif-guided Time Series Counterfactual Explanations [1.1510009152620664]
  We propose a novel model that generates intuitive post-hoc counterfactual explanations. We validated our model using five real-world time-series datasets from the UCR repository.
  arXiv  Detail & Related papers  (2022-11-08T17:56:50Z)
• INFOrmation Prioritization through EmPOWERment in Visual Model-Based RL [90.06845886194235]
  We propose a modified objective for model-based reinforcement learning (RL) We integrate a term inspired by variational empowerment into a state-space model based on mutual information. We evaluate the approach on a suite of vision-based robot control tasks with natural video backgrounds.
  arXiv  Detail & Related papers  (2022-04-18T23:09:23Z)
• Explainable Recommender Systems via Resolving Learning Representations [57.24565012731325]
  Explanations could help improve user experience and discover system defects. We propose a novel explainable recommendation model through improving the transparency of the representation learning process.
  arXiv  Detail & Related papers  (2020-08-21T05:30:48Z)
• Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge [96.92252296244233]
  Large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control. We show that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements. Our work paves a path towards open-domain systems that constantly improve by interacting with users who can instantly correct a model by adding simple natural language statements.
  arXiv  Detail & Related papers  (2020-06-11T17:02:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.