Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence

• URL: http://arxiv.org/abs/2210.07109v1
• Date: Thu, 13 Oct 2022 15:43:39 GMT
• Title: Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence
• Authors: Chris Callison-Burch, Gaurav Singh Tomar, Lara J. Martin, Daphne Ippolito, Suma Bailis, David Reitter
• Abstract summary: We frame D&D as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history. We create a gameplay dataset consisting of nearly 900 games, with a total of 7,000 players, 800,000 dialogue turns, 500,000 dice rolls, and 58 million words. We train a large language model (LM) to generate the next game turn, conditioning it on different information.
• Score: 28.558934742150022
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI researchers have posited Dungeons and Dragons (D&D) as a challenge problem to test systems on various language-related capabilities. In this paper, we frame D&D specifically as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history. We create a gameplay dataset consisting of nearly 900 games, with a total of 7,000 players, 800,000 dialogue turns, 500,000 dice rolls, and 58 million words. We automatically annotate the data with partial state information about the game play. We train a large language model (LM) to generate the next game turn, conditioning it on different information. The LM can respond as a particular character or as the player who runs the game--i.e., the Dungeon Master (DM). It is trained to produce dialogue that is either in-character (roleplaying in the fictional world) or out-of-character (discussing rules or strategy). We perform a human evaluation to determine what factors make the generated output plausible and interesting. We further perform an automatic evaluation to determine how well the model can predict the game state given the history and examine how well tracking the game state improves its ability to produce plausible conversational output.

Related papers

• MCPDial: A Minecraft Persona-driven Dialogue Dataset [22.420926356322568]
  We introduce the Minecraft Persona-driven Dialogue dataset (MCPDial) Starting with a small seed of expert-written conversations, we employ our method to generate hundreds of additional conversations. The conversations are long, allowing for in-depth and extensive interactions between the player and NPC.
  arXiv  Detail & Related papers  (2024-10-29T00:19:55Z)
• Self-Directed Turing Test for Large Language Models [56.64615470513102]
  The Turing test examines whether AIs can exhibit human-like behaviour in natural language conversations. Traditional Turing tests adopt a rigid dialogue format where each participant sends only one message each time. This paper proposes the Self-Directed Turing Test, which extends the original test with a burst dialogue format.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• What if Red Can Talk? Dynamic Dialogue Generation Using Large Language Models [0.0]
  We introduce a dialogue filler framework that utilizes large language models (LLMs) to generate dynamic and contextually appropriate character interactions. We test this framework within the environments of Final Fantasy VII Remake and Pokemon. This study aims to assist developers in crafting more nuanced filler dialogues, thereby enriching player immersion and enhancing the overall RPG experience.
  arXiv  Detail & Related papers  (2024-07-29T19:12:18Z)
• CALYPSO: LLMs as Dungeon Masters' Assistants [46.61924662589895]
  Large language models (LLMs) have shown remarkable abilities to generate coherent natural language text. We introduce CALYPSO, a system of LLM-powered interfaces that support DMs with information and inspiration specific to their own scenario. When given access to CALYPSO, DMs reported that it generated high-fidelity text suitable for direct presentation to players, and low-fidelity ideas that the DM could develop further while maintaining their creative agency.
  arXiv  Detail & Related papers  (2023-08-15T02:57:00Z)
• FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information [75.201485544517]
  We present FIREBALL, a large dataset containing nearly 25,000 unique sessions from real D&D gameplay on Discord with true game state info. We demonstrate that FIREBALL can improve natural language generation (NLG) by using Avrae state information.
  arXiv  Detail & Related papers  (2023-05-02T15:36:10Z)
• Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions [22.669941641551823]
  Large language models (LLMs) such as ChatGPT and GPT-4 have recently demonstrated their remarkable abilities of communicating with human users. We take an initiative to investigate their capacities of playing text games, in which a player has to understand the environment and respond to situations by having dialogues with the game world. Our experiments show that ChatGPT performs competitively compared to all the existing systems but still exhibits a low level of intelligence.
  arXiv  Detail & Related papers  (2023-04-06T05:01:28Z)
• I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons [82.28503603235364]
  We study teacher-student natural language interactions in a goal-driven environment in Dungeons and Dragons. Our approach is to decompose and model these interactions into (1) the Dungeon Master's intent to guide players toward a given goal; (2) the DM's guidance utterance to the players expressing this intent; and (3) a theory-of-mind (ToM) model that anticipates the players' reaction to the guidance one turn into the future.
  arXiv  Detail & Related papers  (2022-12-20T08:06:55Z)
• A Benchmark for Understanding and Generating Dialogue between Characters in Stories [75.29466820496913]
  We present the first study to explore whether machines can understand and generate dialogue in stories. We propose two new tasks including Masked Dialogue Generation and Dialogue Speaker Recognition. We show the difficulty of the proposed tasks by testing existing models with automatic and manual evaluation on DialStory.
  arXiv  Detail & Related papers  (2022-09-18T10:19:04Z)
• DialogZoo: Large-Scale Dialog-Oriented Task Learning [52.18193690394549]
  We aim to build a unified foundation model which can solve massive diverse dialogue tasks. To achieve this goal, we first collect a large-scale well-labeled dialogue dataset from 73 publicly available datasets.
  arXiv  Detail & Related papers  (2022-05-25T11:17:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.