Inherently Explainable Reinforcement Learning in Natural Language
- URL: http://arxiv.org/abs/2112.08907v1
- Date: Thu, 16 Dec 2021 14:24:35 GMT
- Title: Inherently Explainable Reinforcement Learning in Natural Language
- Authors: Xiangyu Peng, Mark O. Riedl, Prithviraj Ammanabrolu
- Abstract summary: We focus on the task of creating a reinforcement learning agent that is inherently explainable.
This Hierarchically Explainable Reinforcement Learning agent operates in Interactive Fictions, text-based game environments.
Our agent is designed to treat explainability as a first-class citizen.
- Score: 14.117921448623342
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We focus on the task of creating a reinforcement learning agent that is
inherently explainable -- with the ability to produce immediate local
explanations by thinking out loud while performing a task and analyzing entire
trajectories post-hoc to produce causal explanations. This Hierarchically
Explainable Reinforcement Learning agent (HEX-RL), operates in Interactive
Fictions, text-based game environments in which an agent perceives and acts
upon the world using textual natural language. These games are usually
structured as puzzles or quests with long-term dependencies in which an agent
must complete a sequence of actions to succeed -- providing ideal environments
in which to test an agent's ability to explain its actions. Our agent is
designed to treat explainability as a first-class citizen, using an extracted
symbolic knowledge graph-based state representation coupled with a Hierarchical
Graph Attention mechanism that points to the facts in the internal graph
representation that most influenced the choice of actions. Experiments show
that this agent provides significantly improved explanations over strong
baselines, as rated by human participants generally unfamiliar with the
environment, while also matching state-of-the-art task performance.
Related papers
- Symbolic Learning Enables Self-Evolving Agents [55.625275970720374]
We introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own.
Agent symbolic learning is designed to optimize the symbolic network within language agents by mimicking two fundamental algorithms in connectionist learning.
We conduct proof-of-concept experiments on both standard benchmarks and complex real-world tasks.
arXiv Detail & Related papers (2024-06-26T17:59:18Z) - On the Effects of Fine-tuning Language Models for Text-Based Reinforcement Learning [19.057241328691077]
We show that rich semantic understanding leads to efficient training of text-based RL agents.
We describe the occurrence of semantic degeneration as a consequence of inappropriate fine-tuning of language models.
arXiv Detail & Related papers (2024-04-15T23:05:57Z) - Explaining Agent Behavior with Large Language Models [7.128139268426959]
We propose an approach to generate natural language explanations for an agent's behavior based only on observations of states and actions.
We show how a compact representation of the agent's behavior can be learned and used to produce plausible explanations.
arXiv Detail & Related papers (2023-09-19T06:13:24Z) - A Closer Look at Reward Decomposition for High-Level Robotic
Explanations [18.019811754800767]
We propose an explainable Q-Map learning framework that combines reward decomposition with abstracted action spaces.
We demonstrate the effectiveness of our framework through quantitative and qualitative analysis of two robotic scenarios.
arXiv Detail & Related papers (2023-04-25T16:01:42Z) - Compositional Generalization in Grounded Language Learning via Induced
Model Sparsity [81.38804205212425]
We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations.
We design an agent that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal.
Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations.
arXiv Detail & Related papers (2022-07-06T08:46:27Z) - Tell me why! -- Explanations support learning of relational and causal
structure [24.434551113103105]
Explanations play a considerable role in human learning, especially in areas that remain major challenges for AI.
We show that reinforcement learning agents might likewise benefit from explanations.
Our results suggest that learning from explanations is a powerful principle that could offer a promising path towards training more robust and general machine learning systems.
arXiv Detail & Related papers (2021-12-07T15:09:06Z) - CausalCity: Complex Simulations with Agency for Causal Discovery and
Reasoning [68.74447489372037]
We present a high-fidelity simulation environment that is designed for developing algorithms for causal discovery and counterfactual reasoning.
A core component of our work is to introduce textitagency, such that it is simple to define and create complex scenarios.
We perform experiments with three state-of-the-art methods to create baselines and highlight the affordances of this environment.
arXiv Detail & Related papers (2021-06-25T00:21:41Z) - Fact-driven Logical Reasoning for Machine Reading Comprehension [82.58857437343974]
We are motivated to cover both commonsense and temporary knowledge clues hierarchically.
Specifically, we propose a general formalism of knowledge units by extracting backbone constituents of the sentence.
We then construct a supergraph on top of the fact units, allowing for the benefit of sentence-level (relations among fact groups) and entity-level interactions.
arXiv Detail & Related papers (2021-05-21T13:11:13Z) - Deep Reinforcement Learning with Stacked Hierarchical Attention for
Text-based Games [64.11746320061965]
We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language.
We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure.
We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
arXiv Detail & Related papers (2020-10-22T12:40:22Z) - Natural Language Rationales with Full-Stack Visual Reasoning: From
Pixels to Semantic Frames to Commonsense Graphs [106.15931418425906]
We present the first study focused on generating natural language rationales across several complex visual reasoning tasks.
We present RationaleVT Transformer, an integrated model that learns to generate free-text rationales by combining pretrained language models with object recognition, grounded visual semantic frames, and visual commonsense graphs.
Our experiments show that the base pretrained language model benefits from visual adaptation and that free-text rationalization is a promising research direction to complement model interpretability for complex visual-textual reasoning tasks.
arXiv Detail & Related papers (2020-10-15T05:08:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.