Turning Mathematics Problems into Games: Reinforcement Learning and Gr\"obner bases together solve Integer Feasibility Problems

• URL: http://arxiv.org/abs/2208.12191v1
• Date: Thu, 25 Aug 2022 16:24:34 GMT
• Title: Turning Mathematics Problems into Games: Reinforcement Learning and Gr\"obner bases together solve Integer Feasibility Problems
• Authors: Yue Wu, Jes\'us A. De Loera
• Abstract summary: We consider the integer feasibility problem, a challenge of deciding whether a system of linear equations and inequalities has a solution with integer values. Our paper describes a novel algebraic reinforcement learning framework that allows an agent to play a game equivalent to the integer feasibility problem. As a proof of concept we demonstrate in experiments that our agent can play well the simplest version of our game for 2-way tables.
• Score: 4.746723775952672
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Can agents be trained to answer difficult mathematical questions by playing a game? We consider the integer feasibility problem, a challenge of deciding whether a system of linear equations and inequalities has a solution with integer values. This is a famous NP-complete problem with applications in many areas of Mathematics and Computer Science. Our paper describes a novel algebraic reinforcement learning framework that allows an agent to play a game equivalent to the integer feasibility problem. We explain how to transform the integer feasibility problem into a game over a set of arrays with fixed margin sums. The game starts with an initial state (an array), and by applying a legal move that leaves the margins unchanged, we aim to eventually reach a winning state with zeros in specific positions. To win the game the player must find a path between the initial state and a final terminal winning state if one exists. Finding such a winning state is equivalent to solving the integer feasibility problem. The key algebraic ingredient is a Gr\"obner basis of the toric ideal for the underlying axial transportation polyhedron. The Gr\"obner basis can be seen as a set of connecting moves (actions) of the game. We then propose a novel RL approach that trains an agent to predict moves in continuous space to cope with the large size of action space. The continuous move is then projected onto the set of legal moves so that the path always leads to valid states. As a proof of concept we demonstrate in experiments that our agent can play well the simplest version of our game for 2-way tables. Our work highlights the potential to train agents to solve non-trivial mathematical queries through contemporary machine learning methods used to train agents to play games.

Related papers

• Imperfect-Recall Games: Equilibrium Concepts and Their Complexity [74.01381499760288]
  We investigate optimal decision making under imperfect recall, that is, when an agent forgets information it once held before. In the framework of extensive-form games with imperfect recall, we analyze the computational complexities of finding equilibria in multiplayer settings.
  arXiv  Detail & Related papers  (2024-06-23T00:27:28Z)
• Neural Population Learning beyond Symmetric Zero-sum Games [52.20454809055356]
  We introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated (CCE) of the game. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality.
  arXiv  Detail & Related papers  (2024-01-10T12:56:24Z)
• Game Solving with Online Fine-Tuning [17.614045403579244]
  This paper investigates applying online fine-tuning while searching and proposes two methods to learn tailor-designed computations for game solving. Our experiments show that using online fine-tuning can solve a series of challenging 7x7 Killall-Go problems, using only 23.54% of time compared to the baseline.
  arXiv  Detail & Related papers  (2023-11-13T09:09:52Z)
• On Grid Graph Reachability and Puzzle Games [0.0]
  Many puzzle video games, like Sokoban, involve moving some agent in a maze. The difficulty of the game is mainly related to performing actions on objects, such as pushing (reachable) boxes. In this paper we study CP and SAT approaches for solving these kind of problems.
  arXiv  Detail & Related papers  (2023-10-02T17:41:35Z)
• Hardness of Independent Learning and Sparse Equilibrium Computation in Markov Games [70.19141208203227]
  We consider the problem of decentralized multi-agent reinforcement learning in Markov games. We show that no algorithm attains no-regret in general-sum games when executed independently by all players. We show that our lower bounds hold even for seemingly easier setting in which all agents are controlled by a centralized algorithm.
  arXiv  Detail & Related papers  (2023-03-22T03:28:12Z)
• Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithms [66.8634598612777]
  We show that Nash equilibrium (NE) is acceptable to all players in a multi-player game. We also show that no one can benefit unilaterally from the general theory step by step.
  arXiv  Detail & Related papers  (2023-01-19T11:36:50Z)
• An AlphaZero-Inspired Approach to Solving Search Problems [63.24965775030674]
  We adapt the methods and techniques used in AlphaZero for solving search problems. We describe possible representations in terms of easy-instance solvers and self-reductions. We also describe a version of Monte Carlo tree search adapted for search problems.
  arXiv  Detail & Related papers  (2022-07-02T23:39:45Z)
• JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding [74.12405417718054]
  This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model(PLM) Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement. We design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
  arXiv  Detail & Related papers  (2022-06-13T17:03:52Z)
• Public Information Representation for Adversarial Team Games [31.29335755664997]
  adversarial team games reside in the asymmetric information available to the team members during the play. Our algorithms convert a sequential team game with adversaries to a classical two-player zero-sum game. Due to the NP-hard nature of the problem, the resulting Public Team game may be exponentially larger than the original one.
  arXiv  Detail & Related papers  (2022-01-25T15:07:12Z)
• Noncommutative Nullstellens\"atze and Perfect Games [0.0]
  The foundations of classical Algebraic Geometry and Real Algebraic Geometry are the Nullsatz and Positivsatz. This paper concerns commuting operator strategies for nonlocal games. Results are spread over different literatures, hence rather than being terse, our style is fairly expository.
  arXiv  Detail & Related papers  (2021-11-29T20:05:33Z)
• Algorithm for Computing Approximate Nash Equilibrium in Continuous Games with Application to Continuous Blotto [1.7132914341329848]
  We present a new algorithm for approximating Nash equilibrium strategies in continuous games. In addition to two-player zero-sum games, our algorithm also applies to multiplayer games and games with imperfect information.
  arXiv  Detail & Related papers  (2020-06-12T19:53:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.