Enhancing Chess Reinforcement Learning with Graph Representation

• URL: http://arxiv.org/abs/2410.23753v1
• Date: Thu, 31 Oct 2024 09:18:47 GMT
• Title: Enhancing Chess Reinforcement Learning with Graph Representation
• Authors: Tomas Rigaux, Hisashi Kashima,
• Abstract summary: We introduce a more general architecture based on Graph Neural Networks (GNN) We show that this new architecture outperforms previous architectures with a similar number of parameters. We also show that the model, when trained on a smaller $5times 5$ variant of chess, is able to be quickly fine-tuned to play on regular $8times 8$ chess.
• Score: 21.919003715442074
• License:
• Abstract: Mastering games is a hard task, as games can be extremely complex, and still fundamentally different in structure from one another. While the AlphaZero algorithm has demonstrated an impressive ability to learn the rules and strategy of a large variety of games, ranging from Go and Chess, to Atari games, its reliance on extensive computational resources and rigid Convolutional Neural Network (CNN) architecture limits its adaptability and scalability. A model trained to play on a $19\times 19$ Go board cannot be used to play on a smaller $13\times 13$ board, despite the similarity between the two Go variants. In this paper, we focus on Chess, and explore using a more generic Graph-based Representation of a game state, rather than a grid-based one, to introduce a more general architecture based on Graph Neural Networks (GNN). We also expand the classical Graph Attention Network (GAT) layer to incorporate edge-features, to naturally provide a generic policy output format. Our experiments, performed on smaller networks than the initial AlphaZero paper, show that this new architecture outperforms previous architectures with a similar number of parameters, being able to increase playing strength an order of magnitude faster. We also show that the model, when trained on a smaller $5\times 5$ variant of chess, is able to be quickly fine-tuned to play on regular $8\times 8$ chess, suggesting that this approach yields promising generalization abilities. Our code is available at https://github.com/akulen/AlphaGateau.

Related papers

• Amortized Planning with Large-Scale Transformers: A Case Study on Chess [11.227110138932442]
  This paper uses chess, a landmark planning problem in AI, to assess performance on a planning task. ChessBench is a large-scale benchmark of 10 million chess games with legal move and value annotations (15 billion points) provided by Stockfish. We show that, although a remarkably good approximation can be distilled into large-scale transformers via supervised learning, perfect distillation is still beyond reach.
  arXiv  Detail & Related papers  (2024-02-07T00:36:24Z)
• From Images to Connections: Can DQN with GNNs learn the Strategic Game of Hex? [22.22813915303447]
  We investigate whether graph neural networks (GNNs) can replace convolutional neural networks (CNNs) in self-play reinforcement learning. GNNs excel at dealing with long range dependency situations in game states and are less prone to overfitting. This suggests a potential paradigm shift, signaling the use of game-specific structures to reshape self-play reinforcement learning.
  arXiv  Detail & Related papers  (2023-11-22T14:20:15Z)
• Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning [86.37438204416435]
  Stratego is one of the few iconic board games that Artificial Intelligence (AI) has not yet mastered. Decisions in Stratego are made over a large number of discrete actions with no obvious link between action and outcome. DeepNash beats existing state-of-the-art AI methods in Stratego and achieved a yearly (2022) and all-time top-3 rank on the Gravon games platform.
  arXiv  Detail & Related papers  (2022-06-30T15:53:19Z)
• Near-Optimal No-Regret Learning for General Convex Games [121.50979258049135]
  We show that regret can be obtained for general convex and compact strategy sets. Our dynamics are on an instantiation of optimistic follow-the-regularized-bounds over an appropriately emphlifted space. Even in those special cases where prior results apply, our algorithm improves over the state-of-the-art regret.
  arXiv  Detail & Related papers  (2022-06-17T12:58:58Z)
• Elastic Monte Carlo Tree Search with State Abstraction for Strategy Game Playing [58.720142291102135]
  Strategy video games challenge AI agents with their search space caused by complex game elements. State abstraction is a popular technique that reduces the state space complexity. We propose Elastic MCTS, an algorithm that uses state abstraction to play strategy games.
  arXiv  Detail & Related papers  (2022-05-30T14:18:45Z)
• Neighbor2Seq: Deep Learning on Massive Graphs by Transforming Neighbors to Sequences [55.329402218608365]
  We propose the Neighbor2Seq to transform the hierarchical neighborhood of each node into a sequence. We evaluate our method on a massive graph with more than 111 million nodes and 1.6 billion edges. Results show that our proposed method is scalable to massive graphs and achieves superior performance across massive and medium-scale graphs.
  arXiv  Detail & Related papers  (2022-02-07T16:38:36Z)
• Train on Small, Play the Large: Scaling Up Board Games with AlphaZero and GNN [23.854093182195246]
  Playing board games is considered a major challenge for both humans and AI researchers. In this work, we look at the board as a graph and combine a graph neural network architecture inside the AlphaZero framework. Our model can be trained quickly to play different challenging board games on multiple board sizes, without using any domain knowledge.
  arXiv  Detail & Related papers  (2021-07-18T08:36:00Z)
• Determining Chess Game State From an Image [19.06796946564999]
  This paper puts forth a new dataset synthesised from a 3D model that is an order of magnitude larger than existing ones. A novel end-to-end chess recognition system is presented that combines traditional computer vision techniques with deep learning. The described system achieves an error rate of 0.23% per square on the test set, 28 times better than the current state of the art.
  arXiv  Detail & Related papers  (2021-04-30T13:02:13Z)
• The Design Of "Stratega": A General Strategy Games Framework [62.997667081978825]
  Stratega is a framework for creating turn-based and real-time strategy games. The framework has been built with a focus on statistical forward planning (SFP) agents. We hope that the development of this framework and its respective agents helps to better understand the complex decision-making process in strategy games.
  arXiv  Detail & Related papers  (2020-09-11T20:02:00Z)
• Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess [5.3524101179510595]
  We use AlphaZero to creatively explore and design new chess variants. We compare nine other variants that involve atomic changes to the rules of chess. By learning near-optimal strategies for each variant with AlphaZero, we determine what games between strong human players might look like if these variants were adopted.
  arXiv  Detail & Related papers  (2020-09-09T15:49:14Z)
• Model-Based Reinforcement Learning for Atari [89.3039240303797]
  We show how video prediction models can enable agents to solve Atari games with fewer interactions than model-free methods. Our experiments evaluate SimPLe on a range of Atari games in low data regime of 100k interactions between the agent and the environment.
  arXiv  Detail & Related papers  (2019-03-01T15:40:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.