Graph Constrained Reinforcement Learning for Natural Language Action Spaces

• URL: http://arxiv.org/abs/2001.08837v1
• Date: Thu, 23 Jan 2020 22:33:18 GMT
• Title: Graph Constrained Reinforcement Learning for Natural Language Action Spaces
• Authors: Prithviraj Ammanabrolu, Matthew Hausknecht
• Abstract summary: Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language. We present KG-A2C, an agent that builds a dynamic knowledge graph while exploring and generates actions using a template-based action space.
• Score: 9.87327247830837
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language. They are ideal environments for studying how to extend reinforcement learning agents to meet the challenges of natural language understanding, partial observability, and action generation in combinatorially-large text-based action spaces. We present KG-A2C, an agent that builds a dynamic knowledge graph while exploring and generates actions using a template-based action space. We contend that the dual uses of the knowledge graph to reason about game state and to constrain natural language generation are the keys to scalable exploration of combinatorially large natural language actions. Results across a wide variety of IF games show that KG-A2C outperforms current IF agents despite the exponential increase in action space size.

Related papers

• LanGWM: Language Grounded World Model [24.86620763902546]
  We focus on learning language-grounded visual features to enhance the world model learning. Our proposed technique of explicit language-grounded visual representation learning has the potential to improve models for human-robot interaction.
  arXiv  Detail & Related papers  (2023-11-29T12:41:55Z)
• Knowledge-enhanced Agents for Interactive Text Games [16.055119735473017]
  We propose a knowledge-injection framework for improved functional grounding of agents in text-based games. We consider two forms of domain knowledge that we inject into learning-based agents: memory of previous correct actions and affordances of relevant objects in the environment. Our framework supports two representative model classes: reinforcement learning agents and language model agents.
  arXiv  Detail & Related papers  (2023-05-08T23:31:39Z)
• A Minimal Approach for Natural Language Action Space in Text-based Games [103.21433712630953]
  This paper revisits the challenge of exploring the action space in text-based games (TGs) We propose $ epsilon$-admissible exploration, a minimal approach of utilizing admissible actions, for training phase. We present a text-based actor-critic (TAC) agent that produces textual commands for game, solely from game observations.
  arXiv  Detail & Related papers  (2023-05-06T16:05:27Z)
• Infusing Commonsense World Models with Graph Knowledge [89.27044249858332]
  We study the setting of generating narratives in an open world text adventure game. A graph representation of the underlying game state can be used to train models that consume and output both grounded graph representations and natural language descriptions and actions.
  arXiv  Detail & Related papers  (2023-01-13T19:58:27Z)
• Learning Action-Effect Dynamics from Pairs of Scene-graphs [50.72283841720014]
  We propose a novel method that leverages scene-graph representation of images to reason about the effects of actions described in natural language. Our proposed approach is effective in terms of performance, data efficiency, and generalization capability compared to existing models.
  arXiv  Detail & Related papers  (2022-12-07T03:36:37Z)
• Learning Knowledge Graph-based World Models of Textual Environments [16.67845396797253]
  This work focuses on the task of building world models of text-based game environments. Our world model learns to simultaneously: (1) predict changes in the world caused by an agent's actions when representing the world as a knowledge graph; and (2) generate the set of contextually relevant natural language actions required to operate in the world.
  arXiv  Detail & Related papers  (2021-06-17T15:45:54Z)
• Situated Language Learning via Interactive Narratives [16.67845396797253]
  This paper explores the question of how to imbue learning agents with the ability to understand and generate contextually relevant natural language. Two key components in creating such agents are interactivity and environment grounding. We discuss the unique challenges a text games' puzzle-like structure combined with natural language state-and-action spaces provides.
  arXiv  Detail & Related papers  (2021-03-18T01:55:16Z)
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games [64.11746320061965]
  We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language. We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
  arXiv  Detail & Related papers  (2020-10-22T12:40:22Z)
• Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning [94.50608198582636]
  Interactive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. We take a novel perspective of IF game solving and re-formulate it as Multi-Passage Reading (MPRC) tasks.
  arXiv  Detail & Related papers  (2020-10-05T23:09:20Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.