Towards a Holodeck-style Simulation Game

• URL: http://arxiv.org/abs/2308.13548v2
• Date: Tue, 12 Sep 2023 10:03:25 GMT
• Title: Towards a Holodeck-style Simulation Game
• Authors: Ahad Shams, Douglas Summers-Stay, Arpan Tripathi, Vsevolod Metelsky, Alexandros Titonis, Karan Malhotra
• Abstract summary: Infinitia uses generative image and language models to reshape all aspects of the setting and NPCs based on a short description from the player. Infinitia is implemented in the Unity engine with a server-client architecture. It uses a multiplayer framework to allow humans to be present and interact in the simulation.
• Score: 40.044978986425676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Infinitia, a simulation game system that uses generative image and language models at play time to reshape all aspects of the setting and NPCs based on a short description from the player, in a way similar to how settings are created on the fictional Holodeck. Building off the ideas of the Generative Agents paper, our system introduces gameplay elements, such as infinite generated fantasy worlds, controllability of NPC behavior, humorous dialogue, cost & time efficiency, collaboration between players and elements of non-determinism among in-game events. Infinitia is implemented in the Unity engine with a server-client architecture, facilitating the addition of exciting features by community developers in the future. Furthermore, it uses a multiplayer framework to allow humans to be present and interact in the simulation. The simulation will be available in open-alpha shortly at https://infinitia.ai/ and we are looking forward to building upon it with the community.

Related papers

• Unbounded: A Generative Infinite Game of Character Life Simulation [68.37260000219479]
  We introduce the concept of a generative infinite game, a video game that transcends the traditional boundaries of finite, hard-coded systems by using generative models. We leverage recent advances in generative AI to create Unbounded: a game of character life simulation that is fully encapsulated in generative models.
  arXiv  Detail & Related papers  (2024-10-24T17:59:31Z)
• A Text-to-Game Engine for UGC-Based Role-Playing Games [6.5715027492220734]
  This paper introduces a new framework for a text-to-game engine that utilizes foundation models to convert simple textual inputs into complex, interactive RPG experiences. The engine dynamically renders the game story in a multi-modal format and adjusts the game character, environment, and mechanics in real-time in response to player actions.
  arXiv  Detail & Related papers  (2024-07-11T05:33:19Z)
• Holodeck: Language Guided Generation of 3D Embodied AI Environments [84.16126434848829]
  Holodeck is a system that generates 3D environments to match a user-supplied prompt fully automatedly. We show that annotators prefer Holodeck over manually designed procedural baselines in residential scenes. We also demonstrate an exciting application of Holodeck in Embodied AI, training agents to navigate in novel scenes without human-constructed data.
  arXiv  Detail & Related papers  (2023-12-14T16:04:14Z)
• Sonicverse: A Multisensory Simulation Platform for Embodied Household Agents that See and Hear [65.33183123368804]
  Sonicverse is a multisensory simulation platform with integrated audio-visual simulation. It enables embodied AI tasks that need audio-visual perception. An agent trained in Sonicverse can successfully perform audio-visual navigation in real-world environments.
  arXiv  Detail & Related papers  (2023-06-01T17:24:01Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)
• Out of the Box: Embodied Navigation in the Real World [45.97756658635314]
  We show how to transfer knowledge acquired in simulation into the real world. We deploy our models on a LoCoBot equipped with a single Intel RealSense camera. Our experiments indicate that it is possible to achieve satisfying results when deploying the obtained model in the real world.
  arXiv  Detail & Related papers  (2021-05-12T18:00:14Z)
• Learning to Simulate Dynamic Environments with GameGAN [109.25308647431952]
  In this paper, we aim to learn a simulator by simply watching an agent interact with an environment. We introduce GameGAN, a generative model that learns to visually imitate a desired game by ingesting screenplay and keyboard actions during training.
  arXiv  Detail & Related papers  (2020-05-25T14:10:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.