Keep CALM and Explore: Language Models for Action Generation in Text-based Games

• URL: http://arxiv.org/abs/2010.02903v1
• Date: Tue, 6 Oct 2020 17:36:29 GMT
• Title: Keep CALM and Explore: Language Models for Action Generation in Text-based Games
• Authors: Shunyu Yao, Rohan Rao, Matthew Hausknecht, Karthik Narasimhan
• Abstract summary: We propose the Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. We combine CALM with a reinforcement learning agent which re-ranks the generated action candidates to maximize in-game rewards.
• Score: 27.00685301984832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. In this paper, we propose the Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. Our key insight is to train language models on human gameplay, where people demonstrate linguistic priors and a general game sense for promising actions conditioned on game history. We combine CALM with a reinforcement learning agent which re-ranks the generated action candidates to maximize in-game rewards. We evaluate our approach using the Jericho benchmark, on games unseen by CALM during training. Our method obtains a 69% relative improvement in average game score over the previous state-of-the-art model. Surprisingly, on half of these games, CALM is competitive with or better than other models that have access to ground truth admissible actions. Code and data are available at https://github.com/princeton-nlp/calm-textgame.

Related papers

• Understanding Players as if They Are Talking to the Game in a Customized Language: A Pilot Study [3.4333699338998693]
  This pilot study explores the application of language models (LMs) to model game event sequences. We transform raw event data into textual sequences and pretraining a Longformer model on this data. The results demonstrate the potential of self-supervised LMs in enhancing game design and personalization without relying on ground-truth labels.
  arXiv  Detail & Related papers  (2024-10-24T09:59:10Z)
• AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game [12.384945632524424]
  This paper focuses on creating proxies of human behavior in simulated environments, with Among Us utilized as a tool for studying simulated human behavior. Our work demonstrates that state-of-the-art large language models (LLMs) can effectively grasp the game rules and make decisions based on the current context.
  arXiv  Detail & Related papers  (2024-07-23T14:34:38Z)
• Efficacy of Language Model Self-Play in Non-Zero-Sum Games [38.644991461153275]
  Game-playing agents like AlphaGo have achieved superhuman performance through self-play. We empirically investigate whether techniques like self-play can effectively be used to improve language models. We find that language model self-play leads to significant performance gains in both cooperation and competition with humans.
  arXiv  Detail & Related papers  (2024-06-27T03:52:35Z)
• Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023 [100.32802766127776]
  Palm is a solution to the Long-Term Action Anticipation task utilizing vision-language and large language models. It predicts future actions based on frame descriptions and action labels extracted from the input videos.
  arXiv  Detail & Related papers  (2023-06-28T20:33:52Z)
• A Minimal Approach for Natural Language Action Space in Text-based Games [103.21433712630953]
  This paper revisits the challenge of exploring the action space in text-based games (TGs) We propose $ epsilon$-admissible exploration, a minimal approach of utilizing admissible actions, for training phase. We present a text-based actor-critic (TAC) agent that produces textual commands for game, solely from game observations.
  arXiv  Detail & Related papers  (2023-05-06T16:05:27Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)
• Infusing Commonsense World Models with Graph Knowledge [89.27044249858332]
  We study the setting of generating narratives in an open world text adventure game. A graph representation of the underlying game state can be used to train models that consume and output both grounded graph representations and natural language descriptions and actions.
  arXiv  Detail & Related papers  (2023-01-13T19:58:27Z)
• Pre-trained Language Models as Prior Knowledge for Playing Text-based Games [2.423547527175808]
  In this paper, we improve the semantic understanding of the agent by proposing a simple RL with LM framework. We perform a detailed study of our framework to demonstrate how our model outperforms all existing agents on the popular game, Zork1. Our proposed approach also performs comparably to the state-of-the-art models on the other set of text games.
  arXiv  Detail & Related papers  (2021-07-18T10:28:48Z)
• Learning Dynamic Belief Graphs to Generalize on Text-Based Games [55.59741414135887]
  Playing text-based games requires skills in processing natural language and sequential decision making. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text.
  arXiv  Detail & Related papers  (2020-02-21T04:38:37Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.