StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games

• URL: http://arxiv.org/abs/2604.25796v1
• Date: Tue, 28 Apr 2026 16:03:14 GMT
• Title: StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games
• Abstract summary: StratFormer is a transformer-based meta-agent that learns to simultaneously model and exploit opponents in imperfect-information games.<n> StratFormer achieves an average exploitation gain of +0.106 BB per hand over GTO, with peak gains of +0.821 against highly exploitable opponents.
• Score: 0.8574682463936006
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present StratFormer, a transformer-based meta-agent that learns to simultaneously model and exploit opponents in imperfect-information games through a two-phase curriculum. The first phase trains an opponent modeling head to identify behavioral patterns from action histories while the agent plays a game-theoretic optimal (GTO) policy. The second phase progressively shifts the policy toward best-response (BR) exploitation, guided by a per-opponent regularization schedule tied to exploitability. Our architecture introduces dual-turn tokens -- feature vectors constructed at both agent and opponent decision points -- coupled with bucket-rate features that encode opponent tendencies across five strategic contexts. On Leduc Hold'em, a small poker variant with six cards and two betting rounds, we test against six opponent archetypes at two strength levels each, with exploitability ranging from 0.15 to 1.26 Big Blinds (BB) per hand. StratFormer achieves an average exploitation gain of +0.106 BB per hand over GTO, with peak gains of +0.821 against highly exploitable opponents, while maintaining near-equilibrium safety.

Related papers

• LLM-Guided Reinforcement Learning for Adaptive NPC Behavior in Multi-Agent Combat Games [0.0]
  We investigate a runtime strategy-selection framework in which a large language model (LLM) guides a trained RL policy without modifying its underlying behavior.<n>We evaluate both configurations against three scripted opponent types across 600 episodes and analyze outcomes using the Mann-Whitney U test.
  arXiv  Detail & Related papers  (2026-08-27T07:17:58Z)
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
  We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
  arXiv  Detail & Related papers  (2026-05-28T07:33:47Z)
• Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games [60.213483076150844]
  We propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL.<n>We conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning.<n>We achieve a 54.76% win rate against GPT-4 in six adversarial games.
  arXiv  Detail & Related papers  (2025-10-19T09:03:28Z)
• Beyond Game Theory Optimal: Profit-Maximizing Poker Agents for No-Limit Holdem [0.06610877051761614]
  Game-Theory- Regret Minimization (CFR) performs best in heads-up situations and CFR remains the strongest method in most multi-way situations.<n>Our approach aims to show how poker agents can move from merely not losing to consistently winning against diverse opponents.
  arXiv  Detail & Related papers  (2025-09-28T08:51:57Z)
• CyGATE: Game-Theoretic Cyber Attack-Defense Engine for Patch Strategy Optimization [73.13843039509386]
  This paper presents CyGATE, a game-theoretic framework modeling attacker-defender interactions.<n>CyGATE frames cyber conflicts as a partially observable game (POSG) across Cyber Kill Chain stages.<n>The framework's flexible architecture enables extension to multi-agent scenarios.
  arXiv  Detail & Related papers  (2025-08-01T09:53:06Z)
• Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
  In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent. We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents. Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
  arXiv  Detail & Related papers  (2024-06-16T15:24:50Z)
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization [57.615269148301515]
  In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
  arXiv  Detail & Related papers  (2023-10-02T08:11:07Z)
• Combining Deep Reinforcement Learning and Search with Generative Models for Game-Theoretic Opponent Modeling [30.465929764202155]
  We introduce a scalable and generic multiagent training regime for opponent modeling using deep game-theoretic reinforcement learning.<n>We first propose Generative Best Respoonse (GenBR), a best response algorithm based on Monte-Carlo Tree Search (MCTS)<n>We use this new method under the framework of Policy Space Response Oracles (PSRO) to automate the generation of an emphoffline opponent model.
  arXiv  Detail & Related papers  (2023-02-01T23:06:23Z)
• Game Theory for Adversarial Attacks and Defenses [0.0]
  Adrial attacks can generate adversarial inputs by applying small but intentionally worst-case perturbations to samples from the dataset. Some adversarial defense techniques are developed to improve the security and robustness of the models and avoid them being attacked.
  arXiv  Detail & Related papers  (2021-10-08T07:38:33Z)
• L2E: Learning to Exploit Your Opponent [66.66334543946672]
  We propose a novel Learning to Exploit framework for implicit opponent modeling. L2E acquires the ability to exploit opponents by a few interactions with different opponents during training. We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically.
  arXiv  Detail & Related papers  (2021-02-18T14:27:59Z)
• Learning to Play Sequential Games versus Unknown Opponents [93.8672371143881]
  We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response.
  arXiv  Detail & Related papers  (2020-07-10T09:33:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.