GAMA: Generative Agents for Multi-Agent Autoformalization

• URL: http://arxiv.org/abs/2412.08805v2
• Date: Tue, 18 Feb 2025 12:06:39 GMT
• Title: GAMA: Generative Agents for Multi-Agent Autoformalization
• Authors: Agnieszka Mensfelt, Kostas Stathis, Vince Trencsenyi,
• Abstract summary: We present a framework that enables the autoformalization of interaction scenarios using agents augmented by large language models (LLMs)<n>The agents translate natural language descriptions of interactions into executable logic programs that define the rules of each game.<n>A tournament simulation then tests the functionality of the generated game rules and strategies.
• Score: 3.5083201638203154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent simulations facilitate the exploration of interactions among both natural and artificial agents. However, modelling real-world scenarios and developing simulations often requires substantial expertise and effort. To streamline this process, we present a framework that enables the autoformalization of interaction scenarios using agents augmented by large language models (LLMs) utilising game-theoretic formalisms. The agents translate natural language descriptions of interactions into executable logic programs that define the rules of each game, ensuring syntactic correctness through validation by a solver. A tournament simulation then tests the functionality of the generated game rules and strategies. After the tournament, if a ground truth payoff matrix is available, an exact semantic validation is performed. We evaluate our approach on a diverse set of 110 natural language descriptions exemplifying five $2\times2$ simultaneous-move games, achieving 100% syntactic and 76.5% semantic correctness in the generated game rules for Claude 3.5 Sonnet, and 99.82% syntactic and 77% semantic correctness for GPT-4o. Additionally, we demonstrate high semantic correctness in autoformalizing gameplay strategies. Overall, the results highlight the potential of autoformalization to leverage LLMs in generating formal reasoning modules for decision-making agents.

Related papers

• FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory [51.96049148869987]
  We present FAIRGAME, a Framework for AI Agents Bias Recognition using Game Theory. We describe its implementation and usage, and we employ it to uncover biased outcomes in popular games among AI agents. Overall, FAIRGAME allows users to reliably and easily simulate their desired games and scenarios.
  arXiv  Detail & Related papers  (2025-04-19T15:29:04Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [94.84458417662404]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios. By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors. LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• Verbalized Bayesian Persuasion [54.55974023595722]
  Information design (ID) explores how a sender influence the optimal behavior of receivers to achieve specific objectives. This work proposes a verbalized framework in Bayesian persuasion (BP), which extends classic BP to real-world games involving human dialogues for the first time. Numerical experiments in dialogue scenarios, such as recommendation letters, courtroom interactions, and law enforcement, validate that our framework can both reproduce theoretical results in classic BP and discover effective persuasion strategies.
  arXiv  Detail & Related papers  (2025-02-03T18:20:10Z)
• Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game [32.791648070823776]
  We propose that language agents can learn through in-context interaction. We develop the Multi-agent Kahneman & Tversky's Optimization (MaKTO) MaKTO achieves a 61% average win rate across various models.
  arXiv  Detail & Related papers  (2025-01-24T04:09:03Z)
• Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent Balderdash [6.65572931991284]
  Large Language Models (LLMs) have shown impressive capabilities in complex tasks and interactive environments. This paper introduces a simulation framework utilizing the game Balderdash to evaluate both the creativity and logical reasoning of LLMs.
  arXiv  Detail & Related papers  (2024-11-15T18:42:48Z)
• Reasoning, Memorization, and Fine-Tuning Language Models for Non-Cooperative Games [18.406992961818368]
  We develop a method that integrates the tree of thoughts and multi-agent framework to enhance the capability of pre-trained language models in solving games. We demonstrate a 65 percent winning rate against benchmark algorithms, with an additional 10 percent improvement after fine-tuning.
  arXiv  Detail & Related papers  (2024-10-18T22:28:22Z)
• Autoformalization of Game Descriptions using Large Language Models [3.5083201638203154]
  We introduce a framework for the autoformalization of game-theoretic scenarios. This translates natural language descriptions into formal logic representations suitable for formal solvers. We evaluate the framework using GPT-4o and a dataset of natural language problem descriptions.
  arXiv  Detail & Related papers  (2024-09-18T20:18:53Z)
• AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game [12.384945632524424]
  This paper focuses on creating proxies of human behavior in simulated environments, with Among Us utilized as a tool for studying simulated human behavior. Our work demonstrates that state-of-the-art large language models (LLMs) can effectively grasp the game rules and make decisions based on the current context.
  arXiv  Detail & Related papers  (2024-07-23T14:34:38Z)
• DrEureka: Language Model Guided Sim-To-Real Transfer [64.14314476811806]
  Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball.
  arXiv  Detail & Related papers  (2024-06-04T04:53:05Z)
• States as Strings as Strategies: Steering Language Models with Game-Theoretic Solvers [44.64118885012762]
  A suitable model of the players, strategies, and payoffs associated with linguistic interactions would enable existing game-theoretic algorithms to provide strategic solutions in the space of language. We present one possible binding from dialogue to game theory as well as generalizations of existing equilibrium finding algorithms to this setting.
  arXiv  Detail & Related papers  (2024-01-24T22:22:00Z)
• ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents [77.34720446306419]
  Alympics is a systematic simulation framework utilizing Large Language Model (LLM) agents for game theory research. Alympics creates a versatile platform for studying complex game theory problems.
  arXiv  Detail & Related papers  (2023-11-06T16:03:46Z)
• Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models [105.39236338147715]
  The paper is inspired by the popular language game Who is Spy'' We develop DEEP to evaluate LLMs' expression and disguising abilities. We then introduce SpyGame, an interactive multi-agent framework.
  arXiv  Detail & Related papers  (2023-10-31T14:37:42Z)
• The Consensus Game: Language Model Generation via Equilibrium Search [73.51411916625032]
  We introduce a new, a training-free, game-theoretic procedure for language model decoding. Our approach casts language model decoding as a regularized imperfect-information sequential signaling game. Applying EQUILIBRIUM-RANKING to LLaMA-7B outperforms the much larger LLaMA-65B and PaLM-540B models.
  arXiv  Detail & Related papers  (2023-10-13T14:27:21Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Pre-trained Language Models as Prior Knowledge for Playing Text-based Games [2.423547527175808]
  In this paper, we improve the semantic understanding of the agent by proposing a simple RL with LM framework. We perform a detailed study of our framework to demonstrate how our model outperforms all existing agents on the popular game, Zork1. Our proposed approach also performs comparably to the state-of-the-art models on the other set of text games.
  arXiv  Detail & Related papers  (2021-07-18T10:28:48Z)
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games [64.11746320061965]
  We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language. We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
  arXiv  Detail & Related papers  (2020-10-22T12:40:22Z)
• Generalization of Agent Behavior through Explicit Representation of Context [14.272883554753323]
  In order to deploy autonomous agents in digital interactive environments, they must be able to act robustly in unseen situations. This paper proposes a principled approach where a context module is coevolved with a skill module in the game. The approach is evaluated in the Flappy Bird and LunarLander video games, as well as in the CARLA autonomous driving simulation.
  arXiv  Detail & Related papers  (2020-06-18T04:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.