Generative Agents for Multi-Agent Autoformalization of Interaction Scenarios

• URL: http://arxiv.org/abs/2412.08805v3
• Date: Thu, 29 May 2025 16:48:53 GMT
• Title: Generative Agents for Multi-Agent Autoformalization of Interaction Scenarios
• Authors: Agnieszka Mensfelt, Kostas Stathis, Vince Trencsenyi,
• Abstract summary: This work introduces the Generative Agents for Multi-Agent Autoformalization (GAMA) framework.<n>GAMA automates the formalization of interaction scenarios in simulations using agents augmented with large language models (LLMs)<n>In experiments with 110 natural language descriptions across five 2x2 simultaneous-move games, GAMA achieves 100% syntactic and 76.5% semantic correctness.
• Score: 3.5083201638203154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent simulations are versatile tools for exploring interactions among natural and artificial agents, but their development typically demands domain expertise and manual effort. This work introduces the Generative Agents for Multi-Agent Autoformalization (GAMA) framework, which automates the formalization of interaction scenarios in simulations using agents augmented with large language models (LLMs). To demonstrate the application of GAMA, we use natural language descriptions of game-theoretic scenarios representing social interactions, and we autoformalize them into executable logic programs defining game rules, with syntactic correctness enforced through a solver-based validation. To ensure runtime validity, an iterative, tournament-based procedure tests the generated rules and strategies, followed by exact semantic validation when ground truth outcomes are available. In experiments with 110 natural language descriptions across five 2x2 simultaneous-move games, GAMA achieves 100% syntactic and 76.5% semantic correctness with Claude 3.5 Sonnet, and 99.82% syntactic and 77% semantic correctness with GPT-4o. The framework also shows high semantic accuracy in autoformalizing agents' strategies.

Related papers

• Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour [26.04296415316974]
  We propose Agentic eXplanations via Interrogative Simulation (AXIS)<n>AXIS generates intelligible causal explanations for pre-trained multi-agent policies.<n>We evaluate AXIS on autonomous driving across 10 scenarios for 5 LLMs.
  arXiv  Detail & Related papers  (2025-05-23T12:19:18Z)
• FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory [51.96049148869987]
  We present FAIRGAME, a Framework for AI Agents Bias Recognition using Game Theory. We describe its implementation and usage, and we employ it to uncover biased outcomes in popular games among AI agents. Overall, FAIRGAME allows users to reliably and easily simulate their desired games and scenarios.
  arXiv  Detail & Related papers  (2025-04-19T15:29:04Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [94.84458417662404]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios. By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors. LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• Verbalized Bayesian Persuasion [54.55974023595722]
  Information design (ID) explores how a sender influence the optimal behavior of receivers to achieve specific objectives. This work proposes a verbalized framework in Bayesian persuasion (BP), which extends classic BP to real-world games involving human dialogues for the first time. Numerical experiments in dialogue scenarios, such as recommendation letters, courtroom interactions, and law enforcement, validate that our framework can both reproduce theoretical results in classic BP and discover effective persuasion strategies.
  arXiv  Detail & Related papers  (2025-02-03T18:20:10Z)
• Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game [32.791648070823776]
  We propose that language agents can learn through in-context interaction. We develop the Multi-agent Kahneman & Tversky's Optimization (MaKTO) MaKTO achieves a 61% average win rate across various models.
  arXiv  Detail & Related papers  (2025-01-24T04:09:03Z)
• DialogAgent: An Auto-engagement Agent for Code Question Answering Data Production [5.030384831047144]
  We present DialogAgent, an automated tool for generating synthetic training data that closely mimics real developer interactions.<n>The tool significantly reduces the reliance on manual data generation, increasing efficiency by 4.8 times compared to traditional methods.
  arXiv  Detail & Related papers  (2024-12-11T03:31:36Z)
• Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent Balderdash [6.65572931991284]
  Large Language Models (LLMs) have shown impressive capabilities in complex tasks and interactive environments. This paper introduces a simulation framework utilizing the game Balderdash to evaluate both the creativity and logical reasoning of LLMs.
  arXiv  Detail & Related papers  (2024-11-15T18:42:48Z)
• Reasoning, Memorization, and Fine-Tuning Language Models for Non-Cooperative Games [18.406992961818368]
  We develop a method that integrates the tree of thoughts and multi-agent framework to enhance the capability of pre-trained language models in solving games. We demonstrate a 65 percent winning rate against benchmark algorithms, with an additional 10 percent improvement after fine-tuning.
  arXiv  Detail & Related papers  (2024-10-18T22:28:22Z)
• Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement [112.04307762405669]
  G"odel Agent is a self-evolving framework inspired by the G"odel machine.<n>G"odel Agent can achieve continuous self-improvement, surpassing manually crafted agents in performance, efficiency, and generalizability.
  arXiv  Detail & Related papers  (2024-10-06T10:49:40Z)
• Autoformalization of Game Descriptions using Large Language Models [3.5083201638203154]
  We introduce a framework for the autoformalization of game-theoretic scenarios. This translates natural language descriptions into formal logic representations suitable for formal solvers. We evaluate the framework using GPT-4o and a dataset of natural language problem descriptions.
  arXiv  Detail & Related papers  (2024-09-18T20:18:53Z)
• AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game [12.384945632524424]
  This paper focuses on creating proxies of human behavior in simulated environments, with Among Us utilized as a tool for studying simulated human behavior. Our work demonstrates that state-of-the-art large language models (LLMs) can effectively grasp the game rules and make decisions based on the current context.
  arXiv  Detail & Related papers  (2024-07-23T14:34:38Z)
• Symbolic Learning Enables Self-Evolving Agents [55.625275970720374]
  We introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own. Agent symbolic learning is designed to optimize the symbolic network within language agents by mimicking two fundamental algorithms in connectionist learning. We conduct proof-of-concept experiments on both standard benchmarks and complex real-world tasks.
  arXiv  Detail & Related papers  (2024-06-26T17:59:18Z)
• DrEureka: Language Model Guided Sim-To-Real Transfer [64.14314476811806]
  Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball.
  arXiv  Detail & Related papers  (2024-06-04T04:53:05Z)
• Procedural Adherence and Interpretability Through Neuro-Symbolic Generative Agents [0.9886108751871757]
  We propose a combination of formal logic-based program synthesis and LLM content generation to bring guarantees of procedural adherence and interpretability to generative agent behavior. To illustrate the benefit of procedural adherence and interpretability, we use Temporal Stream Logic (TSL) to generate an automaton that enforces an interpretable, high-level temporal structure on an agent.
  arXiv  Detail & Related papers  (2024-02-24T21:36:26Z)
• States as Strings as Strategies: Steering Language Models with Game-Theoretic Solvers [44.64118885012762]
  A suitable model of the players, strategies, and payoffs associated with linguistic interactions would enable existing game-theoretic algorithms to provide strategic solutions in the space of language. We present one possible binding from dialogue to game theory as well as generalizations of existing equilibrium finding algorithms to this setting.
  arXiv  Detail & Related papers  (2024-01-24T22:22:00Z)
• Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia [40.82479045442217]
  Generative Agent-Based Models (GABM) are not just classic Agent-Based Models (ABM) GABMs are constructed using an LLM to apply common sense to situations, act "reasonably", recall common semantic knowledge, produce API calls to control digital technologies like apps, and communicate both within the simulation and to researchers viewing it from the outside. Here we present Concordia, a library to facilitate constructing and working with GABMs.
  arXiv  Detail & Related papers  (2023-12-06T18:33:50Z)
• ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents [77.34720446306419]
  Alympics is a systematic simulation framework utilizing Large Language Model (LLM) agents for game theory research. Alympics creates a versatile platform for studying complex game theory problems.
  arXiv  Detail & Related papers  (2023-11-06T16:03:46Z)
• Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models [105.39236338147715]
  The paper is inspired by the popular language game Who is Spy'' We develop DEEP to evaluate LLMs' expression and disguising abilities. We then introduce SpyGame, an interactive multi-agent framework.
  arXiv  Detail & Related papers  (2023-10-31T14:37:42Z)
• The Consensus Game: Language Model Generation via Equilibrium Search [73.51411916625032]
  We introduce a new, a training-free, game-theoretic procedure for language model decoding. Our approach casts language model decoding as a regularized imperfect-information sequential signaling game. Applying EQUILIBRIUM-RANKING to LLaMA-7B outperforms the much larger LLaMA-65B and PaLM-540B models.
  arXiv  Detail & Related papers  (2023-10-13T14:27:21Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• A Model for Intelligible Interaction Between Agents That Predict and Explain [1.335664823620186]
  We formalise the interaction model by taking agents to be automata with some special characteristics. We define One- and Two-Way Intelligibility as properties that emerge at run-time by execution of the protocol. We demonstrate using the formal model to: (a) identify instances of One- and Two-Way Intelligibility in literature reports on humans interacting with ML systems providing logic-based explanations, as is done in Inductive Logic Programming (ILP); and (b) map interactions between humans and machines in an elaborate natural-language based dialogue-model to One- or Two-Way Intellig
  arXiv  Detail & Related papers  (2023-01-04T20:48:22Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Pre-trained Language Models as Prior Knowledge for Playing Text-based Games [2.423547527175808]
  In this paper, we improve the semantic understanding of the agent by proposing a simple RL with LM framework. We perform a detailed study of our framework to demonstrate how our model outperforms all existing agents on the popular game, Zork1. Our proposed approach also performs comparably to the state-of-the-art models on the other set of text games.
  arXiv  Detail & Related papers  (2021-07-18T10:28:48Z)
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games [64.11746320061965]
  We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language. We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
  arXiv  Detail & Related papers  (2020-10-22T12:40:22Z)
• Generalization of Agent Behavior through Explicit Representation of Context [14.272883554753323]
  In order to deploy autonomous agents in digital interactive environments, they must be able to act robustly in unseen situations. This paper proposes a principled approach where a context module is coevolved with a skill module in the game. The approach is evaluated in the Flappy Bird and LunarLander video games, as well as in the CARLA autonomous driving simulation.
  arXiv  Detail & Related papers  (2020-06-18T04:35:22Z)
• SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning [61.335252950832256]
  Sense-Plan-Ask, or SPA, generates plausible verbal interactions between virtual human-like agents and user avatars in shared virtual environments. We find that our algorithm creates a small runtime cost and enables agents to complete their goals more effectively than agents without the ability to leverage natural-language communication.
  arXiv  Detail & Related papers  (2020-02-08T23:15:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.