Balancing of competitive two-player Game Levels with Reinforcement Learning

• URL: http://arxiv.org/abs/2306.04429v1
• Date: Wed, 7 Jun 2023 13:40:20 GMT
• Title: Balancing of competitive two-player Game Levels with Reinforcement Learning
• Authors: Florian Rupp, Manuel Eberhardinger, Kai Eckert
• Abstract summary: We propose an architecture for automated balancing of tile-based levels within the recently introduced PCGRL framework. Our architecture is divided into three parts: (1) a level generator, (2) a balancing agent and, (3) a reward modeling simulation. We show that this approach is capable to teach an agent how to alter a level for balancing better and faster than plain PCGRL.
• Score: 0.2793095554369281
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The balancing process for game levels in a competitive two-player context involves a lot of manual work and testing, particularly in non-symmetrical game levels. In this paper, we propose an architecture for automated balancing of tile-based levels within the recently introduced PCGRL framework (procedural content generation via reinforcement learning). Our architecture is divided into three parts: (1) a level generator, (2) a balancing agent and, (3) a reward modeling simulation. By playing the level in a simulation repeatedly, the balancing agent is rewarded for modifying it towards the same win rates for all players. To this end, we introduce a novel family of swap-based representations to increase robustness towards playability. We show that this approach is capable to teach an agent how to alter a level for balancing better and faster than plain PCGRL. In addition, by analyzing the agent's swapping behavior, we can draw conclusions about which tile types influence the balancing most. We test and show our results using the Neural MMO (NMMO) environment in a competitive two-player setting.

Related papers

• Identifying and Clustering Counter Relationships of Team Compositions in PvP Games for Efficient Balance Analysis [24.683917771144536]
  We develop measures to quantify balance in zero-sum competitive scenarios. We identify useful categories of compositions and pinpoint their counter relationships. Our framework has been validated in popular online games, including Age of Empires II, Hearthstone, Brawl Stars, and League of Legends.
  arXiv  Detail & Related papers  (2024-08-30T10:28:36Z)
• Neural Population Learning beyond Symmetric Zero-sum Games [52.20454809055356]
  We introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated (CCE) of the game. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality.
  arXiv  Detail & Related papers  (2024-01-10T12:56:24Z)
• Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective [80.51463286812314]
  Adversarial Training (AT) has become arguably the state-of-the-art algorithm for extracting robust features. AT suffers from severe robust overfitting problems, particularly after learning rate (LR) decay. We show how LR decay breaks the balance between the minimax game by empowering the trainer with a stronger memorization ability.
  arXiv  Detail & Related papers  (2023-10-30T09:00:11Z)
• Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks [83.28949556413717]
  We study the problem of computing an approximate Nash equilibrium of continuous-action game without access to gradients. We model players' strategies using artificial neural networks. This paper is the first to solve general continuous-action games with unrestricted mixed strategies and without any gradient information.
  arXiv  Detail & Related papers  (2022-11-29T05:16:41Z)
• Three-Player Game Training Dynamics [77.34726150561087]
  We explore three-player game training dynamics using an extended version of a simplified bilinear smooth game. We find that trilinear games do not converge on the Nash equilibrium in most cases. In addition to alternating and simultaneous updates, we explore a new update order--maximizer-first---which is only possible in a three-player game.
  arXiv  Detail & Related papers  (2022-08-12T23:57:44Z)
• Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games [137.86426963572214]
  Turn-based strategy games like Roguelikes, for example, present unique challenges to Deep Reinforcement Learning (DRL) We propose two network architectures to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions.
  arXiv  Detail & Related papers  (2020-12-07T08:47:25Z)
• Multi-Agent Collaboration via Reward Attribution Decomposition [75.36911959491228]
  We propose Collaborative Q-learning (CollaQ) that achieves state-of-the-art performance in the StarCraft multi-agent challenge. CollaQ is evaluated on various StarCraft Attribution maps and shows that it outperforms existing state-of-the-art techniques.
  arXiv  Detail & Related papers  (2020-10-16T17:42:11Z)
• Metagame Autobalancing for Competitive Multiplayer Games [0.10499611180329801]
  We present a tool for balancing multi-player games during game design. Our approach requires a designer to construct an intuitive graphical representation of their meta-game target. We show the capabilities of this tool on examples inheriting from Rock-Paper-Scissors, and on a more complex asymmetric fighting game.
  arXiv  Detail & Related papers  (2020-06-08T08:55:30Z)
• Controllable Level Blending between Games using Variational Autoencoders [6.217860411034386]
  We train a VAE on level data from Super Mario Bros. and Kid Icarus, enabling it to capture the latent space spanning both games. We then use this space to generate level segments that combine properties of levels from both games. We argue that these affordances make the VAE-based approach especially suitable for co-creative level design.
  arXiv  Detail & Related papers  (2020-02-27T01:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.