AlphaExploitem: Going Beyond the Nash Equilibrium in Poker by Learning to Exploit Suboptimal Play

• URL: http://arxiv.org/abs/2605.09150v1
• Date: Sat, 09 May 2026 20:26:35 GMT
• Title: AlphaExploitem: Going Beyond the Nash Equilibrium in Poker by Learning to Exploit Suboptimal Play
• Abstract summary: We introduce AlphaExploitem, which extends the competitive RL poker agent AlphaHoldem by using a hierarchical transformer encoder.<n>We train and evaluate AlphaExploitem on two standard benchmarks for imperfect-information games.
• Score: 0.9558392439655014
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Poker is an imperfect information game that has served as a long-standing benchmark for decision-making under uncertainty. To maximize utility beyond the Nash equilibrium, an agent can deviate from Nash-equilibrium policies to exploit suboptimal play. We introduce AlphaExploitem, which extends the competitive RL poker agent AlphaHoldem by using a hierarchical transformer encoder that enables reasoning over previously played hands and modifying the training procedure with the inclusion of a diverse pool of exploitable opponents to facilitate learning to exploit. We train and evaluate AlphaExploitem on two standard benchmarks for imperfect-information games. Empirically, AlphaExploitem successfully exploits weak play by both in- and out-of-distribution opponents, without losing performance against NE opponents.

Related papers

• CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning [55.425576693143285]
  We introduce CPMbius (CPMobius), a collaborative Coach-Player paradigm for data-free reinforcement learning of reasoning models.<n>Unlike traditional adversarial self-play, CPMbius treats the Coach and Player as independent but cooperative roles.<n>CPMbius achieves substantial improvement without relying on any external training data, outperforming existing unsupervised approaches.
  arXiv  Detail & Related papers  (2026-02-03T01:38:53Z)
• How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use [52.394999779049606]
  Large Language Models (LLMs) are increasingly applied in high-stakes domains.<n>LLMs fail to compete against traditional algorithms.<n>We propose ToolPoker, a tool-integrated reasoning framework.
  arXiv  Detail & Related papers  (2026-01-31T05:45:25Z)
• Who is a Better Player: LLM against LLM [53.46608216197315]
  We propose an adversarial benchmarking framework to assess the comprehensive performance of Large Language Models (LLMs) through board games competition.<n>We introduce Qi Town, a specialized evaluation platform that supports 5 widely played games and involves 20 LLM-driven players.
  arXiv  Detail & Related papers  (2025-08-05T06:41:47Z)
• Mastering Zero-Shot Interactions in Cooperative and Competitive Simultaneous Games [19.46888484570957]
  Albatross is able to exploit weak agents in the competitive game of Battlesnake. It yields an improvement of 37.6% compared to previous state of the art in the cooperative Overcooked benchmark.
  arXiv  Detail & Related papers  (2024-02-05T16:03:44Z)
• Targeted Search Control in AlphaZero for Effective Policy Improvement [93.30151539224144]
  We introduce Go-Exploit, a novel search control strategy for AlphaZero. Go-Exploit samples the start state of its self-play trajectories from an archive of states of interest. Go-Exploit learns with a greater sample efficiency than standard AlphaZero.
  arXiv  Detail & Related papers  (2023-02-23T22:50:24Z)
• Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning [95.78031053296513]
  No-press Diplomacy is a complex strategy game involving both cooperation and competition. We introduce a planning algorithm we call DiL-piKL that regularizes a reward-maximizing policy toward a human imitation-learned policy. We show that DiL-piKL can be extended into a self-play reinforcement learning algorithm we call RL-DiL-piKL.
  arXiv  Detail & Related papers  (2022-10-11T14:47:35Z)
• Anytime Optimal PSRO for Two-Player Zero-Sum Games [17.821479538423155]
  Policy Space Response Oracles (PSRO) is a reinforcement learning algorithm for games that can handle continuous actions. AODO is a double oracle algorithm for 2-player zero-sum games that converges to a Nash equilibrium. We show that our methods achieve far lower exploitability than DO and PSRO and never increase exploitability.
  arXiv  Detail & Related papers  (2022-01-19T16:34:11Z)
• Balancing Adaptability and Non-exploitability in Repeated Games [29.04618208068207]
  We study the problem of guaranteeing low regret in repeated games against an opponent with unknown membership in one of several classes. We add the constraint that our algorithm is non-exploitable, in that the opponent lacks an incentive to use an algorithm against which we cannot achieve rewards exceeding some "fair" value. Our solution is an expert algorithm (LAFF) that searches within a set of sub-algorithms that are optimal for each opponent class and uses a punishment policy upon detecting evidence of exploitation by the opponent.
  arXiv  Detail & Related papers  (2021-12-20T03:09:30Z)
• Discovering Multi-Agent Auto-Curricula in Two-Player Zero-Sum Games [31.97631243571394]
  We introduce a framework, LMAC, that automates the discovery of the update rule without explicit human design. Surprisingly, even without human design, the discovered MARL algorithms achieve competitive or even better performance. We show that LMAC is able to generalise from small games to large games, for example training on Kuhn Poker and outperforming PSRO.
  arXiv  Detail & Related papers  (2021-06-04T22:30:25Z)
• Combining Deep Reinforcement Learning and Search for Imperfect-Information Games [30.520629802135574]
  We present ReBeL, a framework for self-play reinforcement learning and search provably converges to a Nash equilibrium in zero-sum games. We also show ReBeL achieves performance in heads-up no-limit Texas hold'em poker, while using far less domain knowledge than any prior poker AI.
  arXiv  Detail & Related papers  (2020-07-27T15:21:22Z)
• Provable Self-Play Algorithms for Competitive Reinforcement Learning [48.12602400021397]
  We study self-play in competitive reinforcement learning under the setting of Markov games. We show that a self-play algorithm achieves regret $tildemathcalO(sqrtT)$ after playing $T$ steps of the game. We also introduce an explore-then-exploit style algorithm, which achieves a slightly worse regret $tildemathcalO(T2/3)$, but is guaranteed to run in time even in the worst case.
  arXiv  Detail & Related papers  (2020-02-10T18:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.