LLM-Guided Reinforcement Learning for Adaptive NPC Behavior in Multi-Agent Combat Games

• URL: http://arxiv.org/abs/2609.02931v1
• Date: Thu, 27 Aug 2026 07:17:58 GMT
• Title: LLM-Guided Reinforcement Learning for Adaptive NPC Behavior in Multi-Agent Combat Games
• Abstract summary: We investigate a runtime strategy-selection framework in which a large language model (LLM) guides a trained RL policy without modifying its underlying behavior.<n>We evaluate both configurations against three scripted opponent types across 600 episodes and analyze outcomes using the Mann-Whitney U test.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scripted and rule-based non-player characters (NPCs) in combat video games often exhibit predictable behaviors that experienced players can exploit, while reinforcement learning (RL) agents typically retain a fixed policy after training and cannot readily adapt their strategy to different opponents. We investigate a runtime strategy-selection framework in which a large language model (LLM) guides a trained RL policy without modifying its underlying behavior. To demonstrate this, we train five NPC agents with a shared PPO policy in Unity and compare a baseline configuration, in which the policy acts independently, with an LLM-augmented configuration in which a locally hosted Mistral 7B model, accessed through Ollama, reads the live game state every five seconds and assigns one of four tactical tags. We evaluate both configurations against three scripted opponent types across 600 episodes and analyze outcomes using the Mann-Whitney U test. Against a Balanced opponent that changes tactics during an episode, the LLM-augmented agents more than doubled their win rate from 11% to 24% and produced significantly longer episodes. Against an Evasive opponent, the augmented agents achieved a higher win rate and faster kills, although their shorter episode duration did not satisfy the strict hypothesis definition. Against an Aggressive opponent, the LLM's near-constant preference for encirclement was counterproductive. Analysis of 2,430 strategy selections showed that Surround was selected in 83.8% of cases regardless of opponent type, indicating limited zero-shot strategic differentiation at this model scale. These results demonstrate both the potential and limitations of LLM-guided runtime strategy selection for adaptive multi-agent game AI.

Related papers

• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
  We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
  arXiv  Detail & Related papers  (2026-05-28T07:33:47Z)
• StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games [0.8574682463936006]
  StratFormer is a transformer-based meta-agent that learns to simultaneously model and exploit opponents in imperfect-information games.<n> StratFormer achieves an average exploitation gain of +0.106 BB per hand over GTO, with peak gains of +0.821 against highly exploitable opponents.
  arXiv  Detail & Related papers  (2026-04-28T16:03:14Z)
• Learning Controllable and Diverse Player Behaviors in Multi-Agent Environments [0.0]
  This paper introduces a reinforcement learning framework that enables controllable and diverse player behaviors without relying on human gameplay data.<n>We define player behavior in an N-dimensional continuous space and uniformly sample target behavior vectors from a region that encompasses a subset representing real human styles.<n>A single PPO-based multi-agent policy can reproduce new or unseen play styles without retraining.
  arXiv  Detail & Related papers  (2025-12-11T17:26:24Z)
• Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games [60.213483076150844]
  We propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL.<n>We conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning.<n>We achieve a 54.76% win rate against GPT-4 in six adversarial games.
  arXiv  Detail & Related papers  (2025-10-19T09:03:28Z)
• Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies [54.08697738311866]
  Social deduction games like Werewolf combine language, reasoning, and strategy.<n>We curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants.<n>We propose a novel strategy-alignment evaluation that leverages the winning faction's strategies as ground truth in two stages.
  arXiv  Detail & Related papers  (2025-10-13T13:33:30Z)
• PillagerBench: Benchmarking LLM-Based Agents in Competitive Minecraft Team Environments [48.892997022500765]
  We introduce PillagerBench, a framework for evaluating multi-agent systems in real-time competitive team-vs-team scenarios in Minecraft.<n>We also propose TactiCrafter, an LLM-based multi-agent system that facilitates teamwork through human-readable tactics.<n>Our evaluation demonstrates that TactiCrafter outperforms baseline approaches and showcases adaptive learning through self-play.
  arXiv  Detail & Related papers  (2025-09-07T22:51:12Z)
• Preference-based opponent shaping in differentiable games [3.373994463906893]
  We propose a novel Preference-based Opponent Shaping (PBOS) method to enhance the strategy learning process by shaping agents' preferences towards cooperation.<n>We verify the performance of PBOS algorithm in a variety of differentiable games.
  arXiv  Detail & Related papers  (2024-12-04T06:49:21Z)
• Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games [56.70628673595041]
  Large Language Models (LLMs) have been increasingly used in real-world settings, yet their strategic decision-making abilities remain largely unexplored. This work investigates the performance and merits of LLMs in canonical game-theoretic two-player non-zero-sum games, Stag Hunt and Prisoner Dilemma. Our structured evaluation of GPT-3.5, GPT-4-Turbo, GPT-4o, and Llama-3-8B shows that these models, when making decisions in these games, are affected by at least one of the following systematic biases.
  arXiv  Detail & Related papers  (2024-07-05T12:30:02Z)
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization [57.615269148301515]
  In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
  arXiv  Detail & Related papers  (2023-10-02T08:11:07Z)
• L2E: Learning to Exploit Your Opponent [66.66334543946672]
  We propose a novel Learning to Exploit framework for implicit opponent modeling. L2E acquires the ability to exploit opponents by a few interactions with different opponents during training. We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically.
  arXiv  Detail & Related papers  (2021-02-18T14:27:59Z)
• Learning to Model Opponent Learning [11.61673411387596]
  Multi-Agent Reinforcement Learning (MARL) considers settings in which a set of coexisting agents interact with one another and their environment. This poses a great challenge for value function-based algorithms whose convergence usually relies on the assumption of a stationary environment. We develop a novel approach to modelling an opponent's learning dynamics which we term Learning to Model Opponent Learning (LeMOL)
  arXiv  Detail & Related papers  (2020-06-06T17:19:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.