Can Large Language Models Play Text Games Well? Current State-of-the-Art
and Open Questions
- URL: http://arxiv.org/abs/2304.02868v1
- Date: Thu, 6 Apr 2023 05:01:28 GMT
- Title: Can Large Language Models Play Text Games Well? Current State-of-the-Art
and Open Questions
- Authors: Chen Feng Tsai and Xiaochen Zhou and Sierra S. Liu and Jing Li and Mo
Yu and Hongyuan Mei
- Abstract summary: Large language models (LLMs) such as ChatGPT and GPT-4 have recently demonstrated their remarkable abilities of communicating with human users.
We take an initiative to investigate their capacities of playing text games, in which a player has to understand the environment and respond to situations by having dialogues with the game world.
Our experiments show that ChatGPT performs competitively compared to all the existing systems but still exhibits a low level of intelligence.
- Score: 22.669941641551823
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models (LLMs) such as ChatGPT and GPT-4 have recently
demonstrated their remarkable abilities of communicating with human users. In
this technical report, we take an initiative to investigate their capacities of
playing text games, in which a player has to understand the environment and
respond to situations by having dialogues with the game world. Our experiments
show that ChatGPT performs competitively compared to all the existing systems
but still exhibits a low level of intelligence. Precisely, ChatGPT can not
construct the world model by playing the game or even reading the game manual;
it may fail to leverage the world knowledge that it already has; it cannot
infer the goal of each step as the game progresses. Our results open up new
research questions at the intersection of artificial intelligence, machine
learning, and natural language processing.
Related papers
- Open-Ended Wargames with Large Language Models [3.2228025627337864]
We introduce "Snow Globe," an LLM-powered multi-agent system for playing qualitative wargames.
We describe its software architecture conceptually and release an open-source implementation alongside this publication.
We discuss potential applications of the approach and how it fits into the broader wargaming ecosystem.
arXiv Detail & Related papers (2024-04-17T14:54:58Z) - Scaling Instructable Agents Across Many Simulated Worlds [71.1284502230496]
Our goal is to develop an agent that can accomplish anything a human can do in any simulated 3D environment.
Our approach focuses on language-driven generality while imposing minimal assumptions.
Our agents interact with environments in real-time using a generic, human-like interface.
arXiv Detail & Related papers (2024-03-13T17:50:32Z) - A Review of ChatGPT Applications in Education, Marketing, Software
Engineering, and Healthcare: Benefits, Drawbacks, and Research Directions [0.0]
ChatGPT is a type of artificial intelligence language model that uses deep learning algorithms to generate human-like responses to text-based prompts.
The introduction of the latest ChatGPT version in November of 2022 has caused shockwaves in the industrial and academic communities.
arXiv Detail & Related papers (2023-04-29T11:25:43Z) - ChatGPT: Applications, Opportunities, and Threats [0.0]
ChatGPT is an artificial intelligence technology that is fine-tuned using supervised machine learning and reinforcement learning techniques.
The system combines the power of pre-trained deep learning models with a programmability layer to provide a strong base for generating natural language conversations.
Despite its exceptional ability to generate natural-sounding responses, the authors believe that ChatGPT does not possess the same level of understanding, empathy, and creativity as a human.
arXiv Detail & Related papers (2023-04-14T16:25:03Z) - ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large
Language Models in Multilingual Learning [70.57126720079971]
Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP)
This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources.
Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
arXiv Detail & Related papers (2023-04-12T05:08:52Z) - Infusing Commonsense World Models with Graph Knowledge [89.27044249858332]
We study the setting of generating narratives in an open world text adventure game.
A graph representation of the underlying game state can be used to train models that consume and output both grounded graph representations and natural language descriptions and actions.
arXiv Detail & Related papers (2023-01-13T19:58:27Z) - Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence [28.558934742150022]
We frame D&D as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history.
We create a gameplay dataset consisting of nearly 900 games, with a total of 7,000 players, 800,000 dialogue turns, 500,000 dice rolls, and 58 million words.
We train a large language model (LM) to generate the next game turn, conditioning it on different information.
arXiv Detail & Related papers (2022-10-13T15:43:39Z) - Do As I Can, Not As I Say: Grounding Language in Robotic Affordances [119.29555551279155]
Large language models can encode a wealth of semantic knowledge about the world.
Such knowledge could be extremely useful to robots aiming to act upon high-level, temporally extended instructions expressed in natural language.
We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions.
arXiv Detail & Related papers (2022-04-04T17:57:11Z) - AI in (and for) Games [0.9920773256693857]
This chapter outlines the relation between artificial intelligence (AI) / machine learning (ML) algorithms and digital games.
On one hand, AI/ML researchers can generate large, in-the-wild datasets of human affective activity, player behaviour.
On the other hand, games can utilise intelligent algorithms to automate testing of game levels, generate content, develop intelligent and responsive non-player characters (NPCs) or predict and respond player behaviour.
arXiv Detail & Related papers (2021-05-07T08:57:07Z) - Learning Chess Blindfolded: Evaluating Language Models on State Tracking [69.3794549747725]
We consider the task of language modeling for the game of chess.
Unlike natural language, chess notations describe a simple, constrained, and deterministic domain.
We find that transformer language models can learn to track pieces and predict legal moves with high accuracy when trained solely on move sequences.
arXiv Detail & Related papers (2021-02-26T01:16:23Z) - Exploration Based Language Learning for Text-Based Games [72.30525050367216]
This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games.
Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text.
These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
arXiv Detail & Related papers (2020-01-24T03:03:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.