Rotated Bitboards and Reinforcement Learning in Computer Chess and Beyond

• URL: http://arxiv.org/abs/2503.10822v2
• Date: Tue, 08 Apr 2025 19:57:41 GMT
• Title: Rotated Bitboards and Reinforcement Learning in Computer Chess and Beyond
• Authors: Johannes Buchner,
• Abstract summary: We show that the move-generator of FUSc# works 100% correct.<n>We discuss how a "FUSc#-Zero" could be implemented and what would be necessary to reduce the number of training games necessary to achieve a good performance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There exist several techniques for representing the chess board inside the computer. In the first part of this paper, the concepts of the bitboard-representation and the advantages of (rotated) bitboards in move generation are explained. In order to illustrate those ideas practice, the concrete implementation of the move-generator in FUSc# is discussed and we explain a technique how to verify the move-generator with the "perft"-command. We show that the move-generator of FUSc# works 100% correct. The second part of this paper deals with reinforcement learning in computer chess (and beyond). We exemplify the progress that has been made in this field in the last 15-20 years by comparing the "state of the art" from 2002-2008, when FUSc# was developed, with recent innovations connected to "AlphaZero". We discuss how a "FUSc#-Zero" could be implemented and what would be necessary to reduce the number of training games necessary to achieve a good performance. This can be seen as a test case to the general prblem of improving "sample effciency" in reinforcement learning. In the final part, we move beyond computer chess, as the importance of sample effciency extends far beyond board games into a wide range of applications where data is costly, diffcult to obtain, or time consuming to generate. We review some application of the ideas developed in AlphaZero in other domains, i.e. the "other Alphas" like AlphaFold, AlphaTensor, AlphaGeometry and AlphaProof. We also discuss future research and the potential for such methods for ecological economic planning.

Related papers

• Playpen: An Environment for Exploring Learning Through Conversational Interaction [81.67330926729015]
  We look at what extent synthetic interaction in what we call Dialogue Games can provide a learning signal. We investigate the effects of supervised fine-tuning on this data. We release the framework and the baseline training setups in the hope that this can foster research in this promising new direction.
  arXiv  Detail & Related papers  (2025-04-11T14:49:33Z)
• MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games [9.339645051415115]
  MiniZero is a zero-knowledge learning framework that supports four state-of-the-art algorithms. We evaluate the performance of each algorithm in two board games, 9x9 Go and 8x8 Othello, as well as 57 Atari games.
  arXiv  Detail & Related papers  (2023-10-17T14:29:25Z)
• Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning [65.36326734799587]
  We present a novel subgame curriculum learning framework for zero-sum games. It adopts an adaptive initial state distribution by resetting agents to some previously visited states. We derive a subgame selection metric that approximates the squared distance to NE values.
  arXiv  Detail & Related papers  (2023-10-07T13:09:37Z)
• AlphaZero Gomoku [9.434566356382529]
  We broaden the use of AlphaZero to Gomoku, an age-old tactical board game also referred to as "Five in a Row" Our tests demonstrate AlphaZero's versatility in adapting to games other than Go.
  arXiv  Detail & Related papers  (2023-09-04T00:20:06Z)
• Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games [58.720142291102135]
  We describe an effort to add an experimental reinforcement learning system to an existing automated game testing solution based on scripted bots. We show a use-case of leveraging reinforcement learning in game production and cover some of the largest time sinks anyone who wants to make the same journey for their game may encounter. We propose a few research directions that we believe will be valuable and necessary for making machine learning, and especially reinforcement learning, an effective tool in game production.
  arXiv  Detail & Related papers  (2023-07-19T18:19:23Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)
• Targeted Search Control in AlphaZero for Effective Policy Improvement [93.30151539224144]
  We introduce Go-Exploit, a novel search control strategy for AlphaZero. Go-Exploit samples the start state of its self-play trajectories from an archive of states of interest. Go-Exploit learns with a greater sample efficiency than standard AlphaZero.
  arXiv  Detail & Related papers  (2023-02-23T22:50:24Z)
• Neural Networks for Chess [2.055949720959582]
  AlphaZero, Leela Chess Zero and Stockfish NNUE revolutionized Computer Chess. This book gives a complete introduction into the technical inner workings of such engines.
  arXiv  Detail & Related papers  (2022-09-03T22:17:16Z)
• Impartial Games: A Challenge for Reinforcement Learning [0.0]
  We show that AlphaZero-style reinforcement learning algorithms face challenges on impartial games where players share pieces. We show that Nim can be learned on small boards, but the learning progress of AlphaZero-style algorithms dramatically slows down when the board size increases.
  arXiv  Detail & Related papers  (2022-05-25T14:02:02Z)
• Acquisition of Chess Knowledge in AlphaZero [14.41428465712717]
  We show that human knowledge is acquired by the AlphaZero neural network as it trains on the game of chess. By probing for a broad range of human chess concepts we show when and where these concepts are represented in the AlphaZero network. We also provide a behavioural analysis focusing on opening play, including qualitative analysis from chess Grandmaster Vladimir Kramnik.
  arXiv  Detail & Related papers  (2021-11-17T17:46:19Z)
• Combining Off and On-Policy Training in Model-Based Reinforcement Learning [77.34726150561087]
  We propose a way to obtain off-policy targets using data from simulated games in MuZero. Our results show that these targets speed up the training process and lead to faster convergence and higher rewards.
  arXiv  Detail & Related papers  (2021-02-24T10:47:26Z)
• Teach me to play, gamer! Imitative learning in computer games via linguistic description of complex phenomena and decision tree [55.41644538483948]
  We present a new machine learning model by imitation based on the linguistic description of complex phenomena. The method can be a good alternative to design and implement the behaviour of intelligent agents in video game development.
  arXiv  Detail & Related papers  (2021-01-06T21:14:10Z)
• LiveChess2FEN: a Framework for Classifying Chess Pieces based on CNNs [0.0]
  We have implemented a functional framework that automatically digitizes a chess position from an image in less than 1 second. We have analyzed different Convolutional Neural Networks for chess piece classification and how to map them efficiently on our embedded platform.
  arXiv  Detail & Related papers  (2020-12-12T16:48:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.