Playing Text-Based Games with Common Sense

• URL: http://arxiv.org/abs/2012.02757v1
• Date: Fri, 4 Dec 2020 18:22:59 GMT
• Title: Playing Text-Based Games with Common Sense
• Authors: Sahith Dambekodi, Spencer Frazier, Prithviraj Ammanabrolu, Mark O. Riedl
• Abstract summary: We explore two techniques for incorporating commonsense knowledge into agents. Inferring possibly hidden aspects of the world state with either a commonsense inference model COMET, or a language model BERT. We conclude that agents that augment their beliefs about the world state with commonsense inferences are more robust to observational errors and omissions of common elements from text descriptions.
• Score: 12.561548311885671
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text based games are simulations in which an agent interacts with the world purely through natural language. They typically consist of a number of puzzles interspersed with interactions with common everyday objects and locations. Deep reinforcement learning agents can learn to solve these puzzles. However, the everyday interactions with the environment, while trivial for human players, present as additional puzzles to agents. We explore two techniques for incorporating commonsense knowledge into agents. Inferring possibly hidden aspects of the world state with either a commonsense inference model COMET, or a language model BERT. Biasing an agents exploration according to common patterns recognized by a language model. We test our technique in the 9to05 game, which is an extreme version of a text based game that requires numerous interactions with common, everyday objects in common, everyday scenarios. We conclude that agents that augment their beliefs about the world state with commonsense inferences are more robust to observational errors and omissions of common elements from text descriptions.

Related papers

• Scaling Instructable Agents Across Many Simulated Worlds [70.97268311053328]
  Our goal is to develop an agent that can accomplish anything a human can do in any simulated 3D environment. Our approach focuses on language-driven generality while imposing minimal assumptions. Our agents interact with environments in real-time using a generic, human-like interface.
  arXiv  Detail & Related papers  (2024-03-13T17:50:32Z)
• CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents [63.79739920174535]
  We introduce CivRealm, an environment inspired by the Civilization game. CivRealm stands as a unique learning and reasoning challenge for decision-making agents.
  arXiv  Detail & Related papers  (2024-01-19T09:14:11Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models [67.20964015591262]
  We introduce a benchmark named Tachikuma, comprising a Multiple character and novel Object based interaction Estimation task and a supporting dataset. The dataset captures log data from real-time communications during gameplay, providing diverse, grounded, and complex interactions for further explorations. We present a simple prompting baseline and evaluate its performance, demonstrating its effectiveness in enhancing interaction understanding.
  arXiv  Detail & Related papers  (2023-07-24T07:40:59Z)
• Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text [70.14613727284741]
  Communicating with humans is challenging for AIs because it requires a shared understanding of the world, complex semantics, and at times multi-modal gestures. We investigate these challenges in the context of Iconary, a collaborative game of drawing and guessing based on Pictionary. We propose models to play Iconary and train them on over 55,000 games between human players.
  arXiv  Detail & Related papers  (2021-12-01T19:41:03Z)
• Modeling Worlds in Text [16.67845396797253]
  We provide a dataset that enables the creation of learning agents that can build knowledge graph-based world models of interactive narratives. Our dataset provides 24198 mappings between rich natural language observations and knowledge graphs. The training data is collected across 27 games in multiple genres and contains a further 7836 heldout instances over 9 additional games in the test set.
  arXiv  Detail & Related papers  (2021-06-17T15:02:16Z)
• Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents [18.743819704859703]
  It remains unclear to what extent artificial agents utilize semantic understanding of the text. We propose an inverse dynamics decoder to regularize the representation space and encourage exploration. We discuss the implications of our findings for designing future agents with stronger semantic understanding.
  arXiv  Detail & Related papers  (2021-03-25T01:35:27Z)
• A Practical Guide to Studying Emergent Communication through Grounded Language Games [0.0]
  This paper introduces a high-level robot interface that extends the Babel software system. It presents for the first time a toolkit that provides flexible modules for dealing with each subtask involved in running advanced grounded language game experiments.
  arXiv  Detail & Related papers  (2020-04-20T11:48:24Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.