Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games

• URL: http://arxiv.org/abs/2510.16761v1
• Date: Sun, 19 Oct 2025 09:03:28 GMT
• Title: Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games
• Authors: Yikai Zhang, Ye Rong, Siyu Yuan, Jiangjie Chen, Jian Xie, Yanghua Xiao,
• Abstract summary: We propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL.<n>We conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning.<n>We achieve a 54.76% win rate against GPT-4 in six adversarial games.
• Score: 60.213483076150844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing language agents often encounter difficulties in dynamic adversarial games due to poor strategic reasoning. To mitigate this limitation, a promising approach is to allow agents to learn from game interactions automatically, without relying on costly expert-labeled data. Unlike static environments where agents receive fixed feedback or rewards, selecting appropriate opponents in dynamic adversarial games can significantly impact learning performance. However, the discussion of opponents in adversarial environments remains an area under exploration. In this paper, we propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL. Leveraging SCO-PAL, we conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning in such adversarial environments. Utilizing SCO-PAL with self-play, we increase the average win rate against four opponents by approximately 30% compared to baselines and achieve a 54.76% win rate against GPT-4 in six adversarial games.

Related papers

• DiffFP: Learning Behaviors from Scratch via Diffusion-based Fictitious Play [5.8808473430456525]
  We propose DiffFP, a fictitious play framework that estimates the best response to unseen opponents in zero-sum games.<n>We validate our method on complex multi-agent environments, including racing and multi-particle zero-sum games.<n>Our approach achieves up to 3$times$ faster convergence and 30$times$ higher success rates on average against RL-based baselines.
  arXiv  Detail & Related papers  (2025-11-17T09:48:29Z)
• PillagerBench: Benchmarking LLM-Based Agents in Competitive Minecraft Team Environments [48.892997022500765]
  We introduce PillagerBench, a framework for evaluating multi-agent systems in real-time competitive team-vs-team scenarios in Minecraft.<n>We also propose TactiCrafter, an LLM-based multi-agent system that facilitates teamwork through human-readable tactics.<n>Our evaluation demonstrates that TactiCrafter outperforms baseline approaches and showcases adaptive learning through self-play.
  arXiv  Detail & Related papers  (2025-09-07T22:51:12Z)
• Who is a Better Player: LLM against LLM [53.46608216197315]
  We propose an adversarial benchmarking framework to assess the comprehensive performance of Large Language Models (LLMs) through board games competition.<n>We introduce Qi Town, a specialized evaluation platform that supports 5 widely played games and involves 20 LLM-driven players.
  arXiv  Detail & Related papers  (2025-08-05T06:41:47Z)
• Preference-based opponent shaping in differentiable games [3.373994463906893]
  We propose a novel Preference-based Opponent Shaping (PBOS) method to enhance the strategy learning process by shaping agents' preferences towards cooperation.<n>We verify the performance of PBOS algorithm in a variety of differentiable games.
  arXiv  Detail & Related papers  (2024-12-04T06:49:21Z)
• Reinforcement Learning for High-Level Strategic Control in Tower Defense Games [47.618236610219554]
  In strategy games, one of the most important aspects of game design is maintaining a sense of challenge for players. We propose an automated approach that combines traditional scripted methods with reinforcement learning. Results show that combining a learned approach, such as reinforcement learning, with a scripted AI produces a higher-performing and more robust agent than using only AI.
  arXiv  Detail & Related papers  (2024-06-12T08:06:31Z)
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization [57.615269148301515]
  In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
  arXiv  Detail & Related papers  (2023-10-02T08:11:07Z)
• Learning to Play Sequential Games versus Unknown Opponents [93.8672371143881]
  We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response.
  arXiv  Detail & Related papers  (2020-07-10T09:33:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.