Chess Rating Estimation from Moves and Clock Times Using a CNN-LSTM

• URL: http://arxiv.org/abs/2409.11506v2
• Date: Sat, 16 Nov 2024 00:39:44 GMT
• Title: Chess Rating Estimation from Moves and Clock Times Using a CNN-LSTM
• Authors: Michael Omori, Prasad Tadepalli,
• Abstract summary: We propose a method to estimate player ratings directly from game moves and clock times. Our model architecture comprises a CNN to learn positional features, which are integrated with clock-time data into a Bidirectional LSTM. This model is the first to use no hand-crafted features to estimate chess ratings and also the first to output a rating prediction after each move.
• Score: 11.340099493701029
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current chess rating systems update ratings incrementally and may not always accurately reflect a player's true strength at all times, especially for rapidly improving players or very rusty players. To overcome this, we explore a method to estimate player ratings directly from game moves and clock times. We compiled a benchmark dataset from Lichess with over one million games, encompassing various time controls and including move sequences and clock times. Our model architecture comprises a CNN to learn positional features, which are then integrated with clock-time data into a Bidirectional LSTM, predicting player ratings after each move. The model achieved an MAE of 182 rating points on the test data. Additionally, we applied our model to the 2024 IEEE Big Data Cup Chess Puzzle Difficulty Competition dataset, predicted puzzle ratings and achieved competitive results. This model is the first to use no hand-crafted features to estimate chess ratings and also the first to output a rating prediction after each move. Our method highlights the potential of using move-based rating estimation for enhancing rating systems and potentially other applications such as cheating detection.

Related papers

• Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models [0.0]
  The framework employs n-gram language models to capture move patterns characteristic of specific player skill levels.<n>We trained separate models using data from the open-source chess platform Lichess.<n>The framework can classify skill levels with an accuracy of up to 31.7% when utilizing early game information.
  arXiv  Detail & Related papers  (2025-12-01T17:02:07Z)
• Evaluating Language Models' Evaluations of Games [65.49017696754825]
  We advocate for a new paradigm that assesses AI systems' evaluation of games.<n>We leverage a large-scale dataset of over $100$ novel board games and over 450 human judgments.<n>Our results show that reasoning models are generally more aligned to people in their evaluations of games than non-reasoning language models.
  arXiv  Detail & Related papers  (2025-10-13T02:45:37Z)
• Drawing Conclusions from Draws: Rethinking Preference Semantics in Arena-Style LLM Evaluation [17.451562591754698]
  We examine whether a draw genuinely means that the two models are equal.<n>We conjecture that draws are more indicative of query difficulty.<n>We recommend future rating systems to reconsider existing draw semantics.
  arXiv  Detail & Related papers  (2025-10-02T17:59:41Z)
• Online Learning of Counter Categories and Ratings in PvP Games [10.432891534204947]
  We propose an online update algorithm that extends Elo principles to incorporate real-time learning of counter categories. Our method dynamically adjusts both ratings and counter relationships after each match, preserving the explainability of scalar ratings while addressing intransitivity.
  arXiv  Detail & Related papers  (2025-02-06T11:57:18Z)
• Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards [93.16294577018482]
  Arena, the most popular benchmark of this type, ranks models by asking users to select the better response between two randomly selected models. We show that an attacker can alter the leaderboard (to promote their favorite model or demote competitors) at the cost of roughly a thousand votes. Our attack consists of two steps: first, we show how an attacker can determine which model was used to generate a given reply with more than $95%$ accuracy; and then, the attacker can use this information to consistently vote against a target model.
  arXiv  Detail & Related papers  (2025-01-13T17:12:38Z)
• SureMap: Simultaneous Mean Estimation for Single-Task and Multi-Task Disaggregated Evaluation [75.56845750400116]
  Disaggregated evaluation -- estimation of performance of a machine learning model on different subpopulations -- is a core task when assessing performance and group-fairness of AI systems. We develop SureMap that has high estimation accuracy for both multi-task and single-task disaggregated evaluations of blackbox models. Our method combines maximum a posteriori (MAP) estimation using a well-chosen prior together with cross-validation-free tuning via Stein's unbiased risk estimate (SURE)
  arXiv  Detail & Related papers  (2024-11-14T17:53:35Z)
• Analytical and Empirical Study of Herding Effects in Recommendation Systems [72.6693986712978]
  We study how to manage product ratings via rating aggregation rules and shortlisted representative reviews. We show that proper recency aware rating aggregation rules can improve the speed of convergence in Amazon and TripAdvisor.
  arXiv  Detail & Related papers  (2024-08-20T14:29:23Z)
• CNN-based Game State Detection for a Foosball Table [1.612440288407791]
  In the game of Foosball, a compact and comprehensive game state description consists of the positional shifts and rotations of the figures and the position of the ball over time. In this paper, a figure detection system to determine the game state in Foosball is presented. This dataset is utilized to train Convolutional Neural Network (CNN) based end-to-end regression models to predict the rotations and shifts of each rod.
  arXiv  Detail & Related papers  (2024-04-08T09:48:02Z)
• Amortized Planning with Large-Scale Transformers: A Case Study on Chess [11.227110138932442]
  This paper uses chess, a landmark planning problem in AI, to assess performance on a planning task. ChessBench is a large-scale benchmark of 10 million chess games with legal move and value annotations (15 billion points) provided by Stockfish. We show that, although a remarkably good approximation can be distilled into large-scale transformers via supervised learning, perfect distillation is still beyond reach.
  arXiv  Detail & Related papers  (2024-02-07T00:36:24Z)
• End-to-End Chess Recognition [11.15543089335477]
  Current approaches use a pipeline of separate, independent, modules such as chessboard detection, square localization, and piece classification. We explore an end-to-end approach to directly predict the configuration from the image, thus avoiding the error accumulation of the sequential approaches. In contrast to existing datasets that are synthetically rendered and have only limited angles, ChessReD has photographs captured from various angles using smartphone cameras. Our approach in chess recognition on the introduced challenging benchmark dataset outperforms related approaches, successfully recognizing the chess pieces' configuration in 15.26% of ChessReD's test images.
  arXiv  Detail & Related papers  (2023-10-06T08:30:20Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [50.62245481416744]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world. We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique. By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• A Graph-Based Method for Soccer Action Spotting Using Unsupervised Player Classification [75.93186954061943]
  Action spotting involves understanding the dynamics of the game, the complexity of events, and the variation of video sequences. In this work, we focus on the former by (a) identifying and representing the players, referees, and goalkeepers as nodes in a graph, and by (b) modeling their temporal interactions as sequences of graphs. For the player identification task, our method obtains an overall performance of 57.83% average-mAP by combining it with other modalities.
  arXiv  Detail & Related papers  (2022-11-22T15:23:53Z)
• Determining Chess Game State From an Image [19.06796946564999]
  This paper puts forth a new dataset synthesised from a 3D model that is an order of magnitude larger than existing ones. A novel end-to-end chess recognition system is presented that combines traditional computer vision techniques with deep learning. The described system achieves an error rate of 0.23% per square on the test set, 28 times better than the current state of the art.
  arXiv  Detail & Related papers  (2021-04-30T13:02:13Z)
• LiveChess2FEN: a Framework for Classifying Chess Pieces based on CNNs [0.0]
  We have implemented a functional framework that automatically digitizes a chess position from an image in less than 1 second. We have analyzed different Convolutional Neural Networks for chess piece classification and how to map them efficiently on our embedded platform.
  arXiv  Detail & Related papers  (2020-12-12T16:48:40Z)
• Interpretable Real-Time Win Prediction for Honor of Kings, a Popular Mobile MOBA Esport [51.20042288437171]
  We propose a Two-Stage Spatial-Temporal Network (TSSTN) that can provide accurate real-time win predictions. Experiment results and applications in real-world live streaming scenarios showed that the proposed TSSTN model is effective both in prediction accuracy and interpretability.
  arXiv  Detail & Related papers  (2020-08-14T12:00:58Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.