Efficiently Training Neural Networks for Imperfect Information Games by Sampling Information Sets

• URL: http://arxiv.org/abs/2407.05876v1
• Date: Mon, 8 Jul 2024 12:37:07 GMT
• Title: Efficiently Training Neural Networks for Imperfect Information Games by Sampling Information Sets
• Abstract summary: We investigate how a budget of perfect information game evaluations should be distributed among training samples to maximise the return. Our results show that sampling a small number of states, in our experiments roughly 3, for a larger number of separate positions is preferable over repeatedly sampling a smaller quantity of states.
• Score: 4.336779198334903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In imperfect information games, the evaluation of a game state not only depends on the observable world but also relies on hidden parts of the environment. As accessing the obstructed information trivialises state evaluations, one approach to tackle such problems is to estimate the value of the imperfect state as a combination of all states in the information set, i.e., all possible states that are consistent with the current imperfect information. In this work, the goal is to learn a function that maps from the imperfect game information state to its expected value. However, constructing a perfect training set, i.e. an enumeration of the whole information set for numerous imperfect states, is often infeasible. To compute the expected values for an imperfect information game like \textit{Reconnaissance Blind Chess}, one would need to evaluate thousands of chess positions just to obtain the training target for a single state. Still, the expected value of a state can already be approximated with appropriate accuracy from a much smaller set of evaluations. Thus, in this paper, we empirically investigate how a budget of perfect information game evaluations should be distributed among training samples to maximise the return. Our results show that sampling a small number of states, in our experiments roughly 3, for a larger number of separate positions is preferable over repeatedly sampling a smaller quantity of states. Thus, we find that in our case, the quantity of different samples seems to be more important than higher target quality.

Related papers

• MAPLE: Multi-State Aggregated Policy Evaluation for AlphaZero in Imperfect-Information Games [11.496240939986507]
  We propose a tree search method that aggregates policy and value evaluations from multiple sampled world states within a single search tree.<n>Experiments on Phantom Go and Dark Hex show that MAPLE significantly outperforms the PIMC-based AlphaZero baseline.<n>These results demonstrate that MAPLE is an effective approach for AlphaZero-style learning in imperfect-information games.
  arXiv  Detail & Related papers  (2026-05-22T18:59:01Z)
• Neural Network-based Information Set Weighting for Playing Reconnaissance Blind Chess [4.336779198334903]
  In imperfect information games, the game state is generally not fully observable to players. We train two different neural networks which estimate the likelihood of each state in an information set from historical game data. Experimentally, we find that a Siamese neural network is able to achieve higher accuracy and is more efficient than a classical convolutional neural network for the given domain.
  arXiv  Detail & Related papers  (2024-07-08T12:29:29Z)
• CNN-based Game State Detection for a Foosball Table [1.612440288407791]
  In the game of Foosball, a compact and comprehensive game state description consists of the positional shifts and rotations of the figures and the position of the ball over time. In this paper, a figure detection system to determine the game state in Foosball is presented. This dataset is utilized to train Convolutional Neural Network (CNN) based end-to-end regression models to predict the rotations and shifts of each rod.
  arXiv  Detail & Related papers  (2024-04-08T09:48:02Z)
• Visual processing in context of reinforcement learning [0.0]
  This thesis introduces three different representation learning algorithms that have access to different subsets of the data sources that traditional RL algorithms use. We conclude that including unsupervised representation learning in RL problem-solving pipelines can speed up learning.
  arXiv  Detail & Related papers  (2022-08-26T09:30:51Z)
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
  Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings. In this work, we propose a novel sampling-free uncertainty estimation method for object detection. We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
  arXiv  Detail & Related papers  (2021-10-04T17:59:31Z)
• Reasoning-Modulated Representations [85.08205744191078]
  We study a common setting where our task is not purely opaque. Our approach paves the way for a new class of data-efficient representation learning.
  arXiv  Detail & Related papers  (2021-07-19T13:57:13Z)
• Estimating informativeness of samples with Smooth Unique Information [108.25192785062367]
  We measure how much a sample informs the final weights and how much it informs the function computed by the weights. We give efficient approximations of these quantities using a linearized network. We apply these measures to several problems, such as dataset summarization.
  arXiv  Detail & Related papers  (2021-01-17T10:29:29Z)
• When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning? [53.523017945443115]
  We describe natural prediction problems in which every sufficiently accurate training algorithm must encode, in the prediction model, essentially all the information about a large subset of its training examples. Our results do not depend on the training algorithm or the class of models used for learning.
  arXiv  Detail & Related papers  (2020-12-11T15:25:14Z)
• A New Bandit Setting Balancing Information from State Evolution and Corrupted Context [52.67844649650687]
  We propose a new sequential decision-making setting combining key aspects of two established online learning problems with bandit feedback. The optimal action to play at any given moment is contingent on an underlying changing state which is not directly observable by the agent. We present an algorithm that uses a referee to dynamically combine the policies of a contextual bandit and a multi-armed bandit.
  arXiv  Detail & Related papers  (2020-11-16T14:35:37Z)
• JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method [92.15895515035795]
  We introduce a new large scale unconstrained crowd counting dataset (JHU-CROWD++) that contains "4,372" images with "1.51 million" annotations. We propose a novel crowd counting network that progressively generates crowd density maps via residual error estimation.
  arXiv  Detail & Related papers  (2020-04-07T14:59:35Z)
• Towards Using Count-level Weak Supervision for Crowd Counting [55.58468947486247]
  This paper studies the problem of weakly-supervised crowd counting which learns a model from only a small amount of location-level annotations (fully-supervised) but a large amount of count-level annotations (weakly-supervised) We devise a simple-yet-effective training strategy, namely Multiple Auxiliary Tasks Training (MATT), to construct regularizes for restricting the freedom of the generated density maps.
  arXiv  Detail & Related papers  (2020-02-29T02:58:36Z)
• On the consistency of supervised learning with missing values [15.666860186278782]
  In many application settings, the data have missing entries which make analysis challenging. Here, we consider supervised-learning settings: predicting a target when missing values appear in both training and testing data. We show that the widely-used method of imputing with a constant, such as the mean prior to learning, is consistent when missing values are not informative.
  arXiv  Detail & Related papers  (2019-02-19T07:27:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.