Learning to Play Text-based Adventure Games with Maximum Entropy Reinforcement Learning

• URL: http://arxiv.org/abs/2302.10720v2
• Date: Tue, 27 Jun 2023 09:25:32 GMT
• Title: Learning to Play Text-based Adventure Games with Maximum Entropy Reinforcement Learning
• Authors: Weichen Li, Rati Devidze, Sophie Fellenz
• Abstract summary: We adapt the soft-actor-critic (SAC) algorithm to the text-based environment. We show that the reward shaping technique helps the agent to learn the policy faster and achieve higher scores.
• Score: 4.698846136465861
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Text-based games are a popular testbed for language-based reinforcement learning (RL). In previous work, deep Q-learning is commonly used as the learning agent. Q-learning algorithms are challenging to apply to complex real-world domains due to, for example, their instability in training. Therefore, in this paper, we adapt the soft-actor-critic (SAC) algorithm to the text-based environment. To deal with sparse extrinsic rewards from the environment, we combine it with a potential-based reward shaping technique to provide more informative (dense) reward signals to the RL agent. We apply our method to play difficult text-based games. The SAC method achieves higher scores than the Q-learning methods on many games with only half the number of training steps. This shows that it is well-suited for text-based games. Moreover, we show that the reward shaping technique helps the agent to learn the policy faster and achieve higher scores. In particular, we consider a dynamically learned value function as a potential function for shaping the learner's original sparse reward signals.

Related papers

• Autoverse: An Evolvable Game Language for Learning Robust Embodied Agents [2.624282086797512]
  We introduce Autoverse, an evolvable, domain-specific language for single-player 2D grid-based games. We demonstrate its use as a scalable training ground for Open-Ended Learning (OEL) algorithms.
  arXiv  Detail & Related papers  (2024-07-05T02:18:02Z)
• SPRING: Studying the Paper and Reasoning to Play Games [102.5587155284795]
  We propose a novel approach, SPRING, to read the game's original academic paper and use the knowledge learned to reason and play the game through a large language model (LLM) In experiments, we study the quality of in-context "reasoning" induced by different forms of prompts under the setting of the Crafter open-world environment. Our experiments suggest that LLMs, when prompted with consistent chain-of-thought, have great potential in completing sophisticated high-level trajectories.
  arXiv  Detail & Related papers  (2023-05-24T18:14:35Z)
• Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals [69.76245723797368]
  Read and Reward speeds up RL algorithms on Atari games by reading manuals released by the Atari game developers. Various RL algorithms obtain significant improvement in performance and training speed when assisted by our design.
  arXiv  Detail & Related papers  (2023-02-09T05:47:03Z)
• Hierarchical Kickstarting for Skill Transfer in Reinforcement Learning [27.69559938165733]
  Practising and honing skills forms a fundamental component of how humans learn, yet artificial agents are rarely specifically trained to perform them. We investigate how skills can be incorporated into the training of reinforcement learning (RL) agents in complex environments. Our experiments show that learning with a prior knowledge of useful skills can significantly improve the performance of agents on complex problems.
  arXiv  Detail & Related papers  (2022-07-23T19:23:29Z)
• Learning to Shape Rewards using a Game of Switching Controls [21.456451774045465]
  We introduce an automated RS framework in which the shaping-reward function is constructed in a novel game between two agents. We prove theoretically that our framework, which easily adopts existing RL algorithms, learns to construct a shaping-reward function that is tailored to the task. We demonstrate the superior performance of our method against state-of-the-art RS algorithms in Cartpole and the challenging console games Gravitar, Solaris and Super Mario.
  arXiv  Detail & Related papers  (2021-03-16T15:56:57Z)
• An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games [79.23847247132345]
  This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA) We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL)
  arXiv  Detail & Related papers  (2021-01-31T10:30:48Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)
• Learning Dynamic Belief Graphs to Generalize on Text-Based Games [55.59741414135887]
  Playing text-based games requires skills in processing natural language and sequential decision making. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text.
  arXiv  Detail & Related papers  (2020-02-21T04:38:37Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.