Multi-Actor Generative Artificial Intelligence as a Game Engine

• URL: http://arxiv.org/abs/2507.08892v1
• Date: Thu, 10 Jul 2025 22:31:09 GMT
• Title: Multi-Actor Generative Artificial Intelligence as a Game Engine
• Authors: Alexander Sasha Vezhnevets, Jayd Matyas, Logan Cross, Davide Paglieri, Minsuk Chang, William A. Cunningham, Simon Osindero, William S. Isaac, Joel Z. Leibo,
• Abstract summary: Generative AI can be used in multi-actor environments with purposes ranging from social science modeling to interactive narrative and AI evaluation.<n>We argue here that a good approach is to take inspiration from tabletop role-playing games (TTRPGs), where a Game Master (GM) is responsible for the environment and generates all parts of the story not directly determined by the voluntary actions of player characters.
• Score: 49.360775442760314
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI can be used in multi-actor environments with purposes ranging from social science modeling to interactive narrative and AI evaluation. Supporting this diversity of use cases -- which we classify as Simulationist, Dramatist, and Evaluationist -- demands a flexible scenario definition framework. We argue here that a good approach is to take inspiration from tabletop role-playing games (TTRPGs), where a Game Master (GM) is responsible for the environment and generates all parts of the story not directly determined by the voluntary actions of player characters. We argue that the Entity-Component architectural pattern is useful here. In such a system, the GM is not a hardcoded computer game but is itself a configurable entity, composed of components just like any other actor. By design, the approach allows for a separation between the underlying implementation details handled by an engineer, the creation of reusable components, and their composition and configuration managed by a designer who constructs entities from the components. This separation of concerns is instrumental for achieving rapid iteration, maintaining modularity, and ultimately to ensure scalability. We describe the ongoing evolution of the Concordia library in terms of this philosophy, demonstrating how it allows users to effectively configure scenarios that align with their specific goals.

Related papers

• WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents [67.31920821192323]
  We introduce WorldCraft, a system where large language model (LLM) agents leverage procedural generation to create scenes populated with objects.<n>In our framework, a coordinator agent manages the overall process and works with two specialized LLM agents to complete the scene creation.<n>Our pipeline incorporates a trajectory control agent, allowing users to animate the scene and operate the camera through natural language interactions.
  arXiv  Detail & Related papers  (2025-02-21T17:18:30Z)
• DreamGarden: A Designer Assistant for Growing Games from a Single Prompt [1.8906710320196731]
  We present DreamGarden, an AI system capable of assisting with the development of diverse game environments in Unreal Engine. At the core of our method is an LLM-driven planner, capable of breaking down a single, high-level prompt into a hierarchical action plan. This system is presented to the user as a garden of plans and actions, both growing independently and responding to user intervention via seed prompts, pruning, and feedback.
  arXiv  Detail & Related papers  (2024-10-02T17:49:07Z)
• The Drama Machine: Simulating Character Development with LLM Agents [1.999925939110439]
  This paper explores use of multiple large language model (LLM) agents to simulate complex, dynamic characters in dramatic scenarios. We introduce a drama machine framework that coordinates interactions between LLM agents playing different 'Ego' and 'Superego' psychological roles. Results suggest this multi-agent approach can produce more nuanced, adaptive narratives that evolve over a sequence of dialogical turns.
  arXiv  Detail & Related papers  (2024-08-03T09:40:26Z)
• LARP: Language-Agent Role Play for Open-World Games [19.80040627487576]
  Language Agent for Role-Playing (LARP) is a cognitive architecture that encompasses memory processing and a decision-making assistant. The framework refines interactions between users and agents, predefined with unique backgrounds and personalities. It highlights the diverse uses of language models in a range of areas such as entertainment, education, and various simulation scenarios.
  arXiv  Detail & Related papers  (2023-12-24T10:08:59Z)
• CGMI: Configurable General Multi-Agent Interaction Framework [0.0]
  General Multi-Agent Interaction (CGMI) framework designed to replicate human interactions in real-world scenarios. We propose a tree-structured methodology for the assignment, detection, and maintenance of agent personality. We have also integrated general agents to augment the virtual environment's realism.
  arXiv  Detail & Related papers  (2023-08-24T02:03:29Z)
• Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models [67.20964015591262]
  We introduce a benchmark named Tachikuma, comprising a Multiple character and novel Object based interaction Estimation task and a supporting dataset. The dataset captures log data from real-time communications during gameplay, providing diverse, grounded, and complex interactions for further explorations. We present a simple prompting baseline and evaluate its performance, demonstrating its effectiveness in enhancing interaction understanding.
  arXiv  Detail & Related papers  (2023-07-24T07:40:59Z)
• Break and Make: Interactive Structural Understanding Using LEGO Bricks [61.01136603613139]
  We build a fully interactive 3D simulator that allows learning agents to assemble, disassemble and manipulate LEGO models. We take a first step towards solving this problem using sequence-to-sequence models.
  arXiv  Detail & Related papers  (2022-07-27T18:33:09Z)
• Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games [137.86426963572214]
  Turn-based strategy games like Roguelikes, for example, present unique challenges to Deep Reinforcement Learning (DRL) We propose two network architectures to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions.
  arXiv  Detail & Related papers  (2020-12-07T08:47:25Z)
• Learning to Simulate Dynamic Environments with GameGAN [109.25308647431952]
  In this paper, we aim to learn a simulator by simply watching an agent interact with an environment. We introduce GameGAN, a generative model that learns to visually imitate a desired game by ingesting screenplay and keyboard actions during training.
  arXiv  Detail & Related papers  (2020-05-25T14:10:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.