Learning to Follow Instructions in Text-Based Games
- URL: http://arxiv.org/abs/2211.04591v1
- Date: Tue, 8 Nov 2022 22:20:17 GMT
- Title: Learning to Follow Instructions in Text-Based Games
- Authors: Mathieu Tuli, Andrew C. Li, Pashootan Vaezipoor, Toryn Q. Klassen,
Scott Sanner, Sheila A. McIlraith
- Abstract summary: We study the ability of reinforcement learning agents to follow natural language instructions.
We equip RL agents with an internal structured representation of natural language instructions in the form of Linear Temporal Logic.
Our framework both supports and highlights the benefit of understanding the temporal semantics of instructions.
- Score: 30.713430615498375
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Text-based games present a unique class of sequential decision making problem
in which agents interact with a partially observable, simulated environment via
actions and observations conveyed through natural language. Such observations
typically include instructions that, in a reinforcement learning (RL) setting,
can directly or indirectly guide a player towards completing reward-worthy
tasks. In this work, we study the ability of RL agents to follow such
instructions. We conduct experiments that show that the performance of
state-of-the-art text-based game agents is largely unaffected by the presence
or absence of such instructions, and that these agents are typically unable to
execute tasks to completion. To further study and address the task of
instruction following, we equip RL agents with an internal structured
representation of natural language instructions in the form of Linear Temporal
Logic (LTL), a formal language that is increasingly used for temporally
extended reward specification in RL. Our framework both supports and highlights
the benefit of understanding the temporal semantics of instructions and in
measuring progress towards achievement of such a temporally extended behaviour.
Experiments with 500+ games in TextWorld demonstrate the superior performance
of our approach.
Related papers
- STARLING: Self-supervised Training of Text-based Reinforcement Learning Agent with Large Language Models [5.786039929801102]
Existing environments for interactive fiction games are domain-specific or time-consuming to generate and do not train the RL agents to master a specific set of skills.
We introduce an interactive environment for self-supervised RL, STARLING, for text-based games that bootstraps the text-based RL agents with automatically generated games to boost the performance and generalization capabilities to reach a goal of the target environment.
arXiv Detail & Related papers (2024-06-09T18:07:47Z) - On the Effects of Fine-tuning Language Models for Text-Based Reinforcement Learning [19.057241328691077]
We show that rich semantic understanding leads to efficient training of text-based RL agents.
We describe the occurrence of semantic degeneration as a consequence of inappropriate fine-tuning of language models.
arXiv Detail & Related papers (2024-04-15T23:05:57Z) - EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learning [23.83162741035859]
We present EXPLORER, which is an exploration-guided reasoning agent for textual reinforcement learning.
Our experiments show that EXPLORER outperforms the baseline agents on Text-World cooking (TW-Cooking) and Text-World Commonsense (TWC) games.
arXiv Detail & Related papers (2024-03-15T21:22:37Z) - Vision-Language Models Provide Promptable Representations for Reinforcement Learning [67.40524195671479]
We propose a novel approach that uses the vast amounts of general and indexable world knowledge encoded in vision-language models (VLMs) pre-trained on Internet-scale data for embodied reinforcement learning (RL)
We show that our approach can use chain-of-thought prompting to produce representations of common-sense semantic reasoning, improving policy performance in novel scenes by 1.5 times.
arXiv Detail & Related papers (2024-02-05T00:48:56Z) - Learning Symbolic Rules over Abstract Meaning Representations for
Textual Reinforcement Learning [63.148199057487226]
We propose a modular, NEuroSymbolic Textual Agent (NESTA) that combines a generic semantic generalization with a rule induction system to learn interpretable rules as policies.
Our experiments show that the proposed NESTA method outperforms deep reinforcement learning-based techniques by achieving better to unseen test games and learning from fewer training interactions.
arXiv Detail & Related papers (2023-07-05T23:21:05Z) - SPRING: Studying the Paper and Reasoning to Play Games [102.5587155284795]
We propose a novel approach, SPRING, to read the game's original academic paper and use the knowledge learned to reason and play the game through a large language model (LLM)
In experiments, we study the quality of in-context "reasoning" induced by different forms of prompts under the setting of the Crafter open-world environment.
Our experiments suggest that LLMs, when prompted with consistent chain-of-thought, have great potential in completing sophisticated high-level trajectories.
arXiv Detail & Related papers (2023-05-24T18:14:35Z) - Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals [69.76245723797368]
Read and Reward speeds up RL algorithms on Atari games by reading manuals released by the Atari game developers.
Various RL algorithms obtain significant improvement in performance and training speed when assisted by our design.
arXiv Detail & Related papers (2023-02-09T05:47:03Z) - Inherently Explainable Reinforcement Learning in Natural Language [14.117921448623342]
We focus on the task of creating a reinforcement learning agent that is inherently explainable.
This Hierarchically Explainable Reinforcement Learning agent operates in Interactive Fictions, text-based game environments.
Our agent is designed to treat explainability as a first-class citizen.
arXiv Detail & Related papers (2021-12-16T14:24:35Z) - Deep Reinforcement Learning with Stacked Hierarchical Attention for
Text-based Games [64.11746320061965]
We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language.
We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure.
We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
arXiv Detail & Related papers (2020-10-22T12:40:22Z) - Sub-Instruction Aware Vision-and-Language Navigation [46.99329933894108]
Vision-and-language navigation requires an agent to navigate through a real 3D environment following natural language instructions.
We focus on the granularity of the visual and language sequences as well as the traceability of agents through the completion of an instruction.
We propose effective sub-instruction attention and shifting modules that select and attend to a single sub-instruction at each time-step.
arXiv Detail & Related papers (2020-04-06T14:44:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.