Human-aligned Chess with a Bit of Search

• URL: http://arxiv.org/abs/2410.03893v1
• Date: Fri, 4 Oct 2024 19:51:03 GMT
• Title: Human-aligned Chess with a Bit of Search
• Authors: Yiming Zhang, Athul Paul Jacob, Vivian Lai, Daniel Fried, Daphne Ippolito,
• Abstract summary: Chess has long been a testbed for AI's quest to match human intelligence. In this paper, we introduce Allie, a chess-playing AI designed to bridge the gap between artificial and human intelligence in this classic game.
• Score: 35.16633353273246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chess has long been a testbed for AI's quest to match human intelligence, and in recent years, chess AI systems have surpassed the strongest humans at the game. However, these systems are not human-aligned; they are unable to match the skill levels of all human partners or model human-like behaviors beyond piece movement. In this paper, we introduce Allie, a chess-playing AI designed to bridge the gap between artificial and human intelligence in this classic game. Allie is trained on log sequences of real chess games to model the behaviors of human chess players across the skill spectrum, including non-move behaviors such as pondering times and resignations In offline evaluations, we find that Allie exhibits humanlike behavior: it outperforms the existing state-of-the-art in human chess move prediction and "ponders" at critical positions. The model learns to reliably assign reward at each game state, which can be used at inference as a reward function in a novel time-adaptive Monte-Carlo tree search (MCTS) procedure, where the amount of search depends on how long humans would think in the same positions. Adaptive search enables remarkable skill calibration; in a large-scale online evaluation against players with ratings from 1000 to 2600 Elo, our adaptive search method leads to a skill gap of only 49 Elo on average, substantially outperforming search-free and standard MCTS baselines. Against grandmaster-level (2500 Elo) opponents, Allie with adaptive search exhibits the strength of a fellow grandmaster, all while learning exclusively from humans.

Related papers

• Maia-2: A Unified Model for Human-AI Alignment in Chess [10.577896749797485]
  We propose a unified modeling approach for human-AI alignment in chess. We introduce a skill-aware attention mechanism to dynamically integrate players' strengths with encoded chess positions. Our results pave the way for deeper insights into human decision-making and AI-guided teaching tools.
  arXiv  Detail & Related papers  (2024-09-30T17:54:23Z)
• Know your Enemy: Investigating Monte-Carlo Tree Search with Opponent Models in Pommerman [14.668309037894586]
  In combination with Reinforcement Learning, Monte-Carlo Tree Search has shown to outperform human grandmasters in games such as Chess, Shogi and Go. We investigate techniques that transform general-sum multiplayer games into single-player and two-player games.
  arXiv  Detail & Related papers  (2023-05-22T16:39:20Z)
• Are AlphaZero-like Agents Robust to Adversarial Perturbations? [73.13944217915089]
  AlphaZero (AZ) has demonstrated that neural-network-based Go AIs can surpass human performance by a large margin. We ask whether adversarial states exist for Go AIs that may lead them to play surprisingly wrong actions. We develop the first adversarial attack on Go AIs that can efficiently search for adversarial states by strategically reducing the search space.
  arXiv  Detail & Related papers  (2022-11-07T18:43:25Z)
• AI-powered mechanisms as judges: Breaking ties in chess [0.0]
  We propose an AI-driven method for an objective tiebreaking mechanism. The method evaluates the quality of players' moves by comparing them to the optimal moves suggested by powerful chess engines. This approach not only enhances the fairness and integrity of the competition but also maintains the game's high standards.
  arXiv  Detail & Related papers  (2022-10-15T13:27:49Z)
• Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning [95.78031053296513]
  No-press Diplomacy is a complex strategy game involving both cooperation and competition. We introduce a planning algorithm we call DiL-piKL that regularizes a reward-maximizing policy toward a human imitation-learned policy. We show that DiL-piKL can be extended into a self-play reinforcement learning algorithm we call RL-DiL-piKL.
  arXiv  Detail & Related papers  (2022-10-11T14:47:35Z)
• Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning [86.37438204416435]
  Stratego is one of the few iconic board games that Artificial Intelligence (AI) has not yet mastered. Decisions in Stratego are made over a large number of discrete actions with no obvious link between action and outcome. DeepNash beats existing state-of-the-art AI methods in Stratego and achieved a yearly (2022) and all-time top-3 rank on the Gravon games platform.
  arXiv  Detail & Related papers  (2022-06-30T15:53:19Z)
• Collusion Detection in Team-Based Multiplayer Games [57.153233321515984]
  We propose a system that detects colluding behaviors in team-based multiplayer games. The proposed method analyzes the players' social relationships paired with their in-game behavioral patterns. We then automate the detection using Isolation Forest, an unsupervised learning technique specialized in highlighting outliers.
  arXiv  Detail & Related papers  (2022-03-10T02:37:39Z)
• Learning Models of Individual Behavior in Chess [4.793072503820555]
  We develop highly accurate predictive models of individual human behavior in chess. Our work demonstrates a way to bring AI systems into better alignment with the behavior of individual people.
  arXiv  Detail & Related papers  (2020-08-23T18:24:21Z)
• Aligning Superhuman AI with Human Behavior: Chess as a Model System [5.236087378443016]
  We develop Maia, a customized version of Alpha-Zero trained on human chess games, that predicts human moves at a much higher accuracy than existing engines. For a dual task of predicting whether a human will make a large mistake on the next move, we develop a deep neural network that significantly outperforms competitive baselines.
  arXiv  Detail & Related papers  (2020-06-02T18:12:52Z)
• Suphx: Mastering Mahjong with Deep Reinforcement Learning [114.68233321904623]
  We design an AI for Mahjong, named Suphx, based on deep reinforcement learning with some newly introduced techniques. Suphx has demonstrated stronger performance than most top human players in terms of stable rank. This is the first time that a computer program outperforms most top human players in Mahjong.
  arXiv  Detail & Related papers  (2020-03-30T16:18:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.