PureTD: Reinforcement Learning for Backgammon Money Games with No Evaluation-time Search

• URL: http://arxiv.org/abs/2608.15146v1
• Date: Sat, 15 Aug 2026 09:46:03 GMT
• Title: PureTD: Reinforcement Learning for Backgammon Money Games with No Evaluation-time Search
• Abstract summary: We revisit Tesauro's TDGammon for backgammon money games in the setting of no evaluation-time search.<n>Both checker play and cube action are learned from scratch via self-play reinforcement learning (RL), with minimal hand-coded logic and no expert features.
• Score: 51.56484100374058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We revisit Tesauro's TD-Gammon for backgammon money games in the setting of no evaluation-time search. Both checker play and cube action (use of the doubling cube) are learned from scratch via self-play reinforcement learning (RL), with minimal hand-coded logic and no expert features. In this setting, we demonstrate that pure self-play RL suffices to train models that reach near-state-of-the-art playing strength. Specifically, for cubeful money games, our search-free model evaluates faster and is substantially stronger than the open-source engines GNU Backgammon and Open Sage running a one-move (1-ply) look-ahead search.

Related papers

• Adversarial Learning in Games with Bandit Feedback: Logarithmic Pure-Strategy Maximin Regret [64.73231630190121]
  Learning to play zero-sum games is a fundamental problem in game theory and machine learning.<n>We investigate adversarial learning in zero-sum games under bandit feedback, aiming to minimize the deficit against the maximin pure strategy.<n>We show that the Tsallis-INF algorithm achieves $O(c log T)$ instance-dependent regret with a game-dependent parameter $c$.
  arXiv  Detail & Related papers  (2026-02-06T03:26:01Z)
• Can Large Language Models Develop Strategic Reasoning? Post-training Insights from Learning Chess [54.5355907369231]
  We investigate whether large language models (LLMs) can develop strategic reasoning capabilities through reinforcement learning (RL) in chess.<n>Our experiments show that our distillation-based dense rewards often outperform sparse binary rewards.<n>We provide SFT and RL ablations on chess reasoning training and find evidence that this limitation stems from a deficit in the pretrained models' internal understanding of chess.
  arXiv  Detail & Related papers  (2025-07-01T13:16:34Z)
• Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning [89.93384726755106]
  Vision-language reinforcement learning (RL) has primarily focused on narrow domains.<n>We find video games inherently provide rich visual elements and mechanics that are easy to verify.<n>To fully use the multimodal and verifiable reward in video games, we propose Game-RL.
  arXiv  Detail & Related papers  (2025-05-20T03:47:44Z)
• Deep Reinforcement Learning for 5*5 Multiplayer Go [6.222520876209623]
  We propose to use and analyze the latest algorithms that use search and Deep Reinforcement Learning (DRL) We show that using search and DRL we were able to improve the level of play, even though there are more than two players.
  arXiv  Detail & Related papers  (2024-05-23T07:44:24Z)
• Know your Enemy: Investigating Monte-Carlo Tree Search with Opponent Models in Pommerman [14.668309037894586]
  In combination with Reinforcement Learning, Monte-Carlo Tree Search has shown to outperform human grandmasters in games such as Chess, Shogi and Go. We investigate techniques that transform general-sum multiplayer games into single-player and two-player games.
  arXiv  Detail & Related papers  (2023-05-22T16:39:20Z)
• Targeted Search Control in AlphaZero for Effective Policy Improvement [93.30151539224144]
  We introduce Go-Exploit, a novel search control strategy for AlphaZero. Go-Exploit samples the start state of its self-play trajectories from an archive of states of interest. Go-Exploit learns with a greater sample efficiency than standard AlphaZero.
  arXiv  Detail & Related papers  (2023-02-23T22:50:24Z)
• Gym-$\mu$RTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learning [0.0]
  We introduce Gym-$mu$RTS as a fast-to-run RL environment for full-game RTS research. We present a collection of techniques to scale DRL to play full-game $mu$RTS.
  arXiv  Detail & Related papers  (2021-05-21T20:13:35Z)
• Minimax Strikes Back [3.860785927193332]
  Deep Reinforcement Learning reaches a level of play in many complete information games.<n>We take another approach, Ath'enan, which uses a different, Minimax-based, search algorithm called Descent.<n>We show that for multiple games it is much more efficient than the reimplementation of AlphaZero: Polygames.
  arXiv  Detail & Related papers  (2020-12-19T14:42:41Z)
• The NetHack Learning Environment [79.06395964379107]
  We present the NetHack Learning Environment (NLE), a procedurally generated rogue-like environment for Reinforcement Learning research. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration.
  arXiv  Detail & Related papers  (2020-06-24T14:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.