A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games

• URL: http://arxiv.org/abs/2509.03682v1
• Date: Wed, 03 Sep 2025 20:05:58 GMT
• Title: A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games
• Authors: Zhengyang Li, Qijin Ji, Xinghong Ling, Quan Liu,
• Abstract summary: Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated its application potential in modern games.<n>MARL has proven capable of achieving superhuman performance across diverse game environments through techniques like self-play, supervised learning, and deep reinforcement learning.<n>This paper offers insights into MARL in video game AI systems, proposes a novel method to estimate game complexity, and suggests future research directions to advance MARL and its applications in game development.
• Score: 9.115787425836576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated its application potential in modern games. Beginning with foundational work and progressing to landmark achievements such as AlphaStar in StarCraft II and OpenAI Five in Dota 2, MARL has proven capable of achieving superhuman performance across diverse game environments through techniques like self-play, supervised learning, and deep reinforcement learning. With its growing impact, a comprehensive review has become increasingly important in this field. This paper aims to provide a thorough examination of MARL's application from turn-based two-agent games to real-time multi-agent video games including popular genres such as Sports games, First-Person Shooter (FPS) games, Real-Time Strategy (RTS) games and Multiplayer Online Battle Arena (MOBA) games. We further analyze critical challenges posed by MARL in video games, including nonstationary, partial observability, sparse rewards, team coordination, and scalability, and highlight successful implementations in games like Rocket League, Minecraft, Quake III Arena, StarCraft II, Dota 2, Honor of Kings, etc. This paper offers insights into MARL in video game AI systems, proposes a novel method to estimate game complexity, and suggests future research directions to advance MARL and its applications in game development, inspiring further innovation in this rapidly evolving field.

Related papers

• NitroGen: An Open Foundation Model for Generalist Gaming Agents [101.41866522979548]
  NitroGen is a vision-action foundation model for generalist gaming agents.<n>It is trained on 40,000 hours of gameplay videos across more than 1,000 games.
  arXiv  Detail & Related papers  (2026-01-04T16:24:50Z)
• Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning [89.93384726755106]
  Vision-language reinforcement learning (RL) has primarily focused on narrow domains.<n>We find video games inherently provide rich visual elements and mechanics that are easy to verify.<n>To fully use the multimodal and verifiable reward in video games, we propose Game-RL.
  arXiv  Detail & Related papers  (2025-05-20T03:47:44Z)
• AVA: Attentive VLM Agent for Mastering StarCraft II [56.07921367623274]
  We introduce Attentive VLM Agent (AVA), a multimodal StarCraft II agent that aligns artificial agent perception with the human gameplay experience.<n>Our agent addresses this limitation by incorporating RGB visual inputs and natural language observations that more closely simulate human cognitive processes during gameplay.
  arXiv  Detail & Related papers  (2025-03-07T12:54:25Z)
• FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning [25.857375787748715]
  We present FightLadder, a real-time fighting game platform, to empower competitive MARL research. We provide implementations of state-of-the-art MARL algorithms for competitive games, as well as a set of evaluation metrics. We demonstrate the feasibility of this platform by training a general agent that consistently defeats 12 built-in characters in single-player mode.
  arXiv  Detail & Related papers  (2024-06-04T08:04:23Z)
• A Survey on Large Language Model-Based Game Agents [9.140712714337273]
  The development of game agents holds a critical role in advancing towards Artificial General Intelligence.<n>The progress of Large Language Models (LLMs) offers an unprecedented opportunity to evolve and empower game agents.<n>This paper provides a comprehensive overview of LLM-based game agents from a holistic viewpoint.
  arXiv  Detail & Related papers  (2024-04-02T15:34:18Z)
• Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games [58.720142291102135]
  We describe an effort to add an experimental reinforcement learning system to an existing automated game testing solution based on scripted bots. We show a use-case of leveraging reinforcement learning in game production and cover some of the largest time sinks anyone who wants to make the same journey for their game may encounter. We propose a few research directions that we believe will be valuable and necessary for making machine learning, and especially reinforcement learning, an effective tool in game production.
  arXiv  Detail & Related papers  (2023-07-19T18:19:23Z)
• Discovering Multi-Agent Auto-Curricula in Two-Player Zero-Sum Games [31.97631243571394]
  We introduce a framework, LMAC, that automates the discovery of the update rule without explicit human design. Surprisingly, even without human design, the discovered MARL algorithms achieve competitive or even better performance. We show that LMAC is able to generalise from small games to large games, for example training on Kuhn Poker and outperforming PSRO.
  arXiv  Detail & Related papers  (2021-06-04T22:30:25Z)
• DeepCrawl: Deep Reinforcement Learning for Turn-based Strategy Games [137.86426963572214]
  We introduce DeepCrawl, a fully-playable Roguelike prototype for iOS and Android in which all agents are controlled by policy networks trained using Deep Reinforcement Learning (DRL) Our aim is to understand whether recent advances in DRL can be used to develop convincing behavioral models for non-player characters in videogames.
  arXiv  Detail & Related papers  (2020-12-03T13:53:29Z)
• TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning [28.795986840557475]
  TLeague aims at large-scale training and implements several main-stream-MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague.
  arXiv  Detail & Related papers  (2020-11-25T17:24:20Z)
• Reinforcement Learning with Dual-Observation for General Video Game Playing [12.33685708449853]
  General Video Game AI Learning Competition aims to develop agents capable of learning to play different game levels unseen during training. This paper summarises the five years' General Video Game AI Learning Competition editions. We present a novel reinforcement learning technique with dual-observation for general video game playing.
  arXiv  Detail & Related papers  (2020-11-11T08:28:20Z)
• Game-Theoretic Multiagent Reinforcement Learning [73.76806922670379]
  Multiagent reinforcement learning (MARL) corresponds to the learning problem in a multiagent system in which multiple agents learn simultaneously.<n>Despite great successes in MARL, there is a lack of a self-contained overview of the literature that covers game-theoretic foundations of modern MARL methods.<n>This work provides a self-contained assessment of the current state-of-the-art MARL techniques from a game-theoretic perspective.
  arXiv  Detail & Related papers  (2020-11-01T18:24:40Z)
• Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks [48.5733173329785]
  We present Neural MMO, a massively multiagent game environment inspired by MMOs. We discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO.
  arXiv  Detail & Related papers  (2020-01-31T18:50:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.