Are ChatGPT and GPT-4 Good Poker Players? -- A Pre-Flop Analysis

• URL: http://arxiv.org/abs/2308.12466v2
• Date: Thu, 21 Dec 2023 15:14:46 GMT
• Title: Are ChatGPT and GPT-4 Good Poker Players? -- A Pre-Flop Analysis
• Authors: Akshat Gupta
• Abstract summary: We put ChatGPT and GPT-4 through the poker test and evaluate their poker skills. Our findings reveal that while both models display an advanced understanding of poker, both ChatGPT and GPT-4 are NOT game theory optimal poker players.
• Score: 3.4111723103928173
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Since the introduction of ChatGPT and GPT-4, these models have been tested across a large number of tasks. Their adeptness across domains is evident, but their aptitude in playing games, and specifically their aptitude in the realm of poker has remained unexplored. Poker is a game that requires decision making under uncertainty and incomplete information. In this paper, we put ChatGPT and GPT-4 through the poker test and evaluate their poker skills. Our findings reveal that while both models display an advanced understanding of poker, encompassing concepts like the valuation of starting hands, playing positions and other intricacies of game theory optimal (GTO) poker, both ChatGPT and GPT-4 are NOT game theory optimal poker players. Profitable strategies in poker are evaluated in expectations over large samples. Through a series of experiments, we first discover the characteristics of optimal prompts and model parameters for playing poker with these models. Our observations then unveil the distinct playing personas of the two models. We first conclude that GPT-4 is a more advanced poker player than ChatGPT. This exploration then sheds light on the divergent poker tactics of the two models: ChatGPT's conservativeness juxtaposed against GPT-4's aggression. In poker vernacular, when tasked to play GTO poker, ChatGPT plays like a nit, which means that it has a propensity to only engage with premium hands and folds a majority of hands. When subjected to the same directive, GPT-4 plays like a maniac, showcasing a loose and aggressive style of play. Both strategies, although relatively advanced, are not game theory optimal.

Related papers

• Instruction-Driven Game Engine: A Poker Case Study [53.689520884467065]
  The IDGE project aims to democratize game development by enabling a large language model to follow free-form game descriptions and generate game-play processes. We train the IDGE in a curriculum manner that progressively increases its exposure to complex scenarios. Our initial progress lies in developing an IDGE for Poker, which not only supports a wide range of poker variants but also allows for highly individualized new poker games through natural language inputs.
  arXiv  Detail & Related papers  (2024-10-17T11:16:27Z)
• Show, Don't Tell: Evaluating Large Language Models Beyond Textual Understanding with ChildPlay [0.0]
  We use games like Tic-Tac-Toe, Connect Four, and Battleship to assess strategic thinking and decision-making. Despite their proficiency on standard benchmarks, GPT-3.5 and GPT-4's abilities to play and reason about fully observable games without pre-training is mediocre.
  arXiv  Detail & Related papers  (2024-07-12T14:17:26Z)
• PokerGPT: An End-to-End Lightweight Solver for Multi-Player Texas Hold'em via Large Language Model [14.14786217204364]
  Poker, also known as Texas Hold'em, has always been a typical research target within imperfect information games (IIGs) We introduce PokerGPT, an end-to-end solver for playing Texas Hold'em with arbitrary number of players and gaining high win rates.
  arXiv  Detail & Related papers  (2024-01-04T13:27:50Z)
• A Survey on Game Theory Optimal Poker [0.0]
  No non-trivial imperfect information game has been solved to date. This makes poker a great test bed for Artificial Intelligence research. We discuss the intricacies of abstraction techniques, betting models, and specific strategies employed by successful poker bots.
  arXiv  Detail & Related papers  (2024-01-02T04:19:25Z)
• Recording and Describing Poker Hands [40.39759037668144]
  Poker lacks a consistent format that humans can use to document poker hands across different variants. We propose the PHH format which provides a concise human-readable machine-friendly representation of hand history. In the supplementary, we provide 10,088 hands covering 11 different variants in the PHH format.
  arXiv  Detail & Related papers  (2023-12-18T23:39:01Z)
• PokerKit: A Comprehensive Python Library for Fine-Grained Multi-Variant Poker Game Simulations [40.39759037668144]
  PokerKit is an open-source Python library designed to overcome the restrictions of existing poker game simulation and hand evaluation tools. It supports an extensive array of poker variants and provides a flexible architecture for users to define their custom games. The flexibility of PokerKit allows for applications in diverse areas, such as poker AI development, tool creation, and online poker casino implementation.
  arXiv  Detail & Related papers  (2023-08-08T13:54:48Z)
• Sparks of Artificial General Intelligence: Early experiments with GPT-4 [66.1188263570629]
  GPT-4, developed by OpenAI, was trained using an unprecedented scale of compute and data. We demonstrate that GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more. We believe GPT-4 could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system.
  arXiv  Detail & Related papers  (2023-03-22T16:51:28Z)
• Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT [103.57103957631067]
  ChatGPT has attracted great attention, as it can generate fluent and high-quality responses to human inquiries. We evaluate ChatGPT's understanding ability by evaluating it on the most popular GLUE benchmark, and comparing it with 4 representative fine-tuned BERT-style models. We find that: 1) ChatGPT falls short in handling paraphrase and similarity tasks; 2) ChatGPT outperforms all BERT models on inference tasks by a large margin; 3) ChatGPT achieves comparable performance compared with BERT on sentiment analysis and question answering tasks.
  arXiv  Detail & Related papers  (2023-02-19T12:29:33Z)
• Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning [86.37438204416435]
  Stratego is one of the few iconic board games that Artificial Intelligence (AI) has not yet mastered. Decisions in Stratego are made over a large number of discrete actions with no obvious link between action and outcome. DeepNash beats existing state-of-the-art AI methods in Stratego and achieved a yearly (2022) and all-time top-3 rank on the Gravon games platform.
  arXiv  Detail & Related papers  (2022-06-30T15:53:19Z)
• Principal Trade-off Analysis [79.16635054977068]
  We show "Principal Trade-off Analysis" (PTA), a decomposition method that embeds games into a low-dimensional feature space. PTA represents an arbitrary two-player zero-sum game as the weighted sum of pairs of 2D feature planes. We demonstrate the validity of PTA on a quartet of games (Kuhn poker, RPS+2, Blotto, and Pokemon)
  arXiv  Detail & Related papers  (2022-06-09T18:16:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.