Reproducing AlphaZero on Tablut: Self-Play RL for an Asymmetric Board Game

• URL: http://arxiv.org/abs/2604.05476v1
• Date: Tue, 07 Apr 2026 06:16:19 GMT
• Title: Reproducing AlphaZero on Tablut: Self-Play RL for an Asymmetric Board Game
• Abstract summary: This work investigates the adaptation of the AlphaZero reinforcement learning algorithm to Tablut, an asymmetric historical board game.<n>Over 100 self-play iterations, the modified model demonstrated steady improvement, achieving a BayesElo rating of 1235.<n>Ultimately, the experiments confirm that AlphaZero's self-play framework can transfer to highly asymmetric games.
• Score: 0.6574517227976925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work investigates the adaptation of the AlphaZero reinforcement learning algorithm to Tablut, an asymmetric historical board game featuring unequal piece counts and distinct player objectives (king capture versus king escape). While the original AlphaZero architecture successfully leverages a single policy and value head for symmetric games, applying it to asymmetric environments forces the network to learn two conflicting evaluation functions, which can hinder learning efficiency and performance. To address this, the core architecture is modified to use separate policy and value heads for each player role, while maintaining a shared residual trunk to learn common board features. During training, the asymmetric structure introduced training instabilities, notably catastrophic forgetting between the attacker and defender roles. These issues were mitigated by applying C4 data augmentation, increasing the replay buffer size, and having the model play 25 percent of training games against randomly sampled past checkpoints. Over 100 self-play iterations, the modified model demonstrated steady improvement, achieving a BayesElo rating of 1235 relative to a randomly initialized baseline. Training metrics also showed a significant decrease in policy entropy and average remaining pieces, reflecting increasingly focused and decisive play. Ultimately, the experiments confirm that AlphaZero's self-play framework can transfer to highly asymmetric games, provided that distinct policy/value heads and robust stabilization techniques are employed.

Related papers

• Parametric Open Source Games [78.29352305480285]
  We introduce parametric open-source games, a continuous analogue of program equilibria.<n>We derive an exact coupling threshold at which selfish gradient ascent in symmetric $2times2$ games switches from defection toward cooperation.<n>We extend the framework to a neural semantics class whose first-order cooperation condition is governed by the ratio of cross-player to self-player sensitivity.
  arXiv  Detail & Related papers  (2026-06-25T14:14:24Z)
• The Attacker in the Mirror: Breaking Self-Consistency in Safety via Anchored Bipolicy Self-Play [16.696570190611112]
  Self-play red team is an established approach to improving AI safety.<n>We propose Anchored Bipolicy Self-Play, which trains role-specific LoRA adapters on top of a frozen base model.<n>We show up to 100x greater parameter efficiency than finetuning and consistent improvements in safety compared to self-play fine-tuned models.
  arXiv  Detail & Related papers  (2026-05-08T19:41:59Z)
• MindGames Arena Generalization Track: In2AI Solution with Delayed Per-Step Reward Attribution [45.957713330643934]
  Training language model agents for multi-agent strategic interaction presents a core difficulty.<n>Standard reinforcement learning assumes that rewards can be assigned at each step, but this assumption fails in settings where outcomes are entangled across time and agents.<n>We introduce delayed per-step reward attribution with eligibility gating, an episode lifecycle and postprocessing pipeline.
  arXiv  Detail & Related papers  (2026-04-13T23:59:03Z)
• Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training [63.34044358216334]
  ACTOR-CURATOR is a scalable and fully automated curriculum learning framework for reinforcement learning post-training of large language models.<n> Empirically, ACTOR-CURATOR consistently outperforms uniform sampling and strong curriculum baselines.
  arXiv  Detail & Related papers  (2026-02-24T04:19:48Z)
• Learning Tennis Strategy Through Curriculum-Based Dueling Double Deep Q-Networks [0.3181222613501246]
  Tennis strategy optimization is a challenging sequential decision-making problem involving hierarchical scoring, outcomes, long-horizon credit assignment, physical fatigue, and adaptation to opponent skill.<n>I present a reinforcement learning framework that integrates a custom tennis simulation environment with a Dueling Double Deep Q-Network trained using curriculum learning.
  arXiv  Detail & Related papers  (2025-12-20T04:22:07Z)
• Yahtzee: Reinforcement Learning Techniques for Stochastic Combinatorial Games [0.0]
  We formulate an optimal policy for solitaire Yahtzee using Markov Decision Process (MDP) methods.<n>We train self-play agents using various policy gradient methods.<n>All models struggle to learn the upper bonus strategy, overindexing on four-of-a-kind's.
  arXiv  Detail & Related papers  (2025-12-18T20:03:32Z)
• Aspiration-based Perturbed Learning Automata in Games with Noisy Utility Measurements. Part A: Stochastic Stability in Non-zero-Sum Games [0.0]
  This paper introduces a payoff-based learning scheme for distributed optimization, namely aspiration-based learning automata (APLA)<n>We provide a stability analysis of APLA in multi-player positive-utility games under the presence of noisy observations.
  arXiv  Detail & Related papers  (2025-10-31T18:17:38Z)
• Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR [110.90317717368264]
  We propose an online Self-play with Variational problem Synthesis (SvS) strategy for RLVR training.<n>This strategy effectively maintains policy entropy during training and substantially improves Pass@k compared with standard RLVR.
  arXiv  Detail & Related papers  (2025-08-19T17:42:45Z)
• Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Reward [85.84943447589511]
  This paper introduces AC3 (Actor-Critic for Continuous Chunks), a novel RL framework that learns to generate high-dimensional, continuous action sequences.<n>To make this learning process stable and data-efficient, AC3 incorporates targeted stabilization mechanisms for both the actor and the critic.
  arXiv  Detail & Related papers  (2025-08-15T01:27:15Z)
• SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning [27.20778530252474]
  SPIRAL is a self-play framework where models learn by playing multi-turn, zero-sum games against continuously improving versions of themselves.<n>Using SPIRAL, self-play on zero-sum games produces reasoning capabilities that transfer broadly.<n>Analysis reveals that this transfer occurs through three cognitive patterns: systematic decomposition, expected value calculation, and case-by-case analysis.
  arXiv  Detail & Related papers  (2025-06-30T17:58:13Z)
• Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models [64.47869632167284]
  Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities.<n>This sequential approach creates a mismatch -- attackers overfit to obsolete defenses, while defenders perpetually lag behind emerging threats.<n>We propose Self-RedTeam, an online self-play reinforcement learning algorithm where an attacker and defender agent co-evolve through continuous interaction.
  arXiv  Detail & Related papers  (2025-06-09T06:35:12Z)
• Robust Reinforcement Learning using Adversarial Populations [118.73193330231163]
  Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness. We show that using a single adversary does not consistently yield robustness to dynamics variations under standard parametrizations of the adversary. We propose a population-based augmentation to the Robust RL formulation in which we randomly initialize a population of adversaries and sample from the population uniformly during training.
  arXiv  Detail & Related papers  (2020-08-04T20:57:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.