ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic
Decision-Making with AI Agents
- URL: http://arxiv.org/abs/2311.03220v4
- Date: Tue, 16 Jan 2024 07:12:32 GMT
- Title: ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic
Decision-Making with AI Agents
- Authors: Shaoguang Mao, Yuzhe Cai, Yan Xia, Wenshan Wu, Xun Wang, Fengyi Wang,
Tao Ge, Furu Wei
- Abstract summary: Alympics is a systematic simulation framework utilizing Large Language Model (LLM) agents for game theory research.
Alympics creates a versatile platform for studying complex game theory problems.
- Score: 77.34720446306419
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper introduces Alympics (Olympics for Agents), a systematic simulation
framework utilizing Large Language Model (LLM) agents for game theory research.
Alympics creates a versatile platform for studying complex game theory
problems, bridging the gap between theoretical game theory and empirical
investigations by providing a controlled environment for simulating human-like
strategic interactions with LLM agents. In our pilot case study, the "Water
Allocation Challenge," we explore Alympics through a challenging strategic game
focused on the multi-round auction on scarce survival resources. This study
demonstrates the framework's ability to qualitatively and quantitatively
analyze game determinants, strategies, and outcomes. Additionally, we conduct a
comprehensive human assessment and an in-depth evaluation of LLM agents in
strategic decision-making scenarios. Our findings not only expand the
understanding of LLM agents' proficiency in emulating human strategic behavior
but also highlight their potential in advancing game theory knowledge, thereby
enriching our understanding of both game theory and empowering further research
into strategic decision-making domains with LLM agents. Codes, prompts, and all
related resources are available at https://github.com/microsoft/Alympics.
Related papers
- Large Language Models Playing Mixed Strategy Nash Equilibrium Games [1.060608983034705]
This paper focuses on the capabilities of Large Language Models to find the Nash equilibrium in games with a mixed strategy Nash equilibrium and no pure strategy Nash equilibrium.
The study reveals a significant enhancement in the performance of LLMs when they are equipped with the possibility to run code.
It is evident that while LLMs exhibit remarkable proficiency in well-known standard games, their performance dwindles when faced with slight modifications of the same games.
arXiv Detail & Related papers (2024-06-15T09:30:20Z) - GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations [87.99872683336395]
Large Language Models (LLMs) are integrated into critical real-world applications.
This paper evaluates LLMs' reasoning abilities in competitive environments.
We first propose GTBench, a language-driven environment composing 10 widely recognized tasks.
arXiv Detail & Related papers (2024-02-19T18:23:36Z) - Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach [7.693497788883165]
Large language model (LLM) agents, such as Voyage and MetaGPT, present the immense potential in solving intricate tasks.
We propose a Chain of Summarization method, including single frame summarization for processing raw observations and multi frame summarization for analyzing game information.
Experiment results demonstrate that: 1. LLMs possess the relevant knowledge and complex planning abilities needed to address StarCraft II scenarios; 2. Human experts consider the performance of LLM agents to be close to that of an average player who has played StarCraft II for eight years; 3. LLM agents are capable of defeating the built in AI
arXiv Detail & Related papers (2023-12-19T05:27:16Z) - Leveraging Word Guessing Games to Assess the Intelligence of Large
Language Models [105.39236338147715]
The paper is inspired by the popular language game Who is Spy''
We develop DEEP to evaluate LLMs' expression and disguising abilities.
We then introduce SpyGame, an interactive multi-agent framework.
arXiv Detail & Related papers (2023-10-31T14:37:42Z) - Language Agents with Reinforcement Learning for Strategic Play in the
Werewolf Game [40.438765131992525]
We develop strategic language agents that generate flexible language actions and possess strong decision-making abilities.
To mitigate the intrinsic bias in language actions, our agents use an LLM to perform deductive reasoning and generate a diverse set of action candidates.
Experiments show that our agents overcome the intrinsic bias and outperform existing LLM-based agents in the Werewolf game.
arXiv Detail & Related papers (2023-10-29T09:02:57Z) - LLM-Based Agent Society Investigation: Collaboration and Confrontation
in Avalon Gameplay [57.202649879872624]
We present a novel framework designed to seamlessly adapt to Avalon gameplay.
The core of our proposed framework is a multi-agent system that enables efficient communication and interaction among agents.
Our results demonstrate the effectiveness of our framework in generating adaptive and intelligent agents.
arXiv Detail & Related papers (2023-10-23T14:35:26Z) - The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI)
We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents.
We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
arXiv Detail & Related papers (2023-09-14T17:12:03Z) - Strategic Reasoning with Language Models [35.63300060111918]
Strategic reasoning enables agents to cooperate, communicate, and compete with other agents in diverse situations.
Existing approaches to solving strategic games rely on extensive training, yielding strategies that do not generalize to new scenarios or games without retraining.
This paper introduces an approach that uses pretrained Large Language Models with few-shot chain-of-thought examples to enable strategic reasoning for AI agents.
arXiv Detail & Related papers (2023-05-30T16:09:19Z) - SPRING: Studying the Paper and Reasoning to Play Games [102.5587155284795]
We propose a novel approach, SPRING, to read the game's original academic paper and use the knowledge learned to reason and play the game through a large language model (LLM)
In experiments, we study the quality of in-context "reasoning" induced by different forms of prompts under the setting of the Crafter open-world environment.
Our experiments suggest that LLMs, when prompted with consistent chain-of-thought, have great potential in completing sophisticated high-level trajectories.
arXiv Detail & Related papers (2023-05-24T18:14:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.