Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game

• URL: http://arxiv.org/abs/2501.14225v2
• Date: Thu, 13 Mar 2025 03:55:17 GMT
• Title: Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game
• Authors: Rong Ye, Yongxin Zhang, Yikai Zhang, Haoyu Kuang, Zhongyu Wei, Peng Sun,
• Abstract summary: We propose that language agents can learn through in-context interaction.<n>We develop the Multi-agent Kahneman & Tversky's Optimization (MaKTO)<n>MaKTO achieves a 61% average win rate across various models.
• Score: 32.791648070823776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Achieving Artificial General Intelligence (AGI) requires AI agents that can not only make stratigic decisions but also engage in flexible and meaningful communication. Inspired by Wittgenstein's language game theory in Philosophical Investigations, we propose that language agents can learn through in-context interaction rather than traditional multi-stage frameworks that separate decision-making from language expression. Using Werewolf, a social deduction game that tests language understanding, strategic interaction, and adaptability, we develop the Multi-agent Kahneman & Tversky's Optimization (MaKTO). MaKTO engages diverse models in extensive gameplay to generate unpaired desirable and unacceptable responses, then employs KTO to refine the model's decision-making process. In 9-player Werewolf games, MaKTO achieves a 61% average win rate across various models, outperforming GPT-4o and two-stage RL agents by relative improvements of 23.0% and 10.9%, respectively. Notably, MaKTO also demonstrates human-like performance, winning 60% against expert players and showing only 49% detectability in Turing-style blind tests.

Related papers

• FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory [51.96049148869987]
  We present FAIRGAME, a Framework for AI Agents Bias Recognition using Game Theory. We describe its implementation and usage, and we employ it to uncover biased outcomes in popular games among AI agents. Overall, FAIRGAME allows users to reliably and easily simulate their desired games and scenarios.
  arXiv  Detail & Related papers  (2025-04-19T15:29:04Z)
• MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation [60.52580061637301]
  MMLU-ProX is a comprehensive benchmark covering 13 typologically diverse languages with approximately 11,829 questions per language. We evaluate 25 state-of-the-art large language models (LLMs) using 5-shot chain-of-thought (CoT) and zero-shot prompting strategies, analyzing their performance across linguistic and cultural boundaries. Our experiments reveal consistent performance degradation from high-resource languages to lower-resource ones, with the best models achieving over 70% accuracy on English but dropping to around 40% for languages like Swahili.
  arXiv  Detail & Related papers  (2025-03-13T15:59:20Z)
• Learning Strategic Language Agents in the Werewolf Game with Iterative Latent Space Policy Optimization [13.496120603859701]
  Large language model (LLM)-based agents have recently shown impressive progress in a variety of domains. Applying these agents to social deduction games such as Werewolf, which requires both strategic decision-making and free-form language interaction, remains non-trivial. We propose Latent Space Policy Optimization (LSPO), an iterative framework that addresses these challenges by first mapping free-form text to a discrete latent space.
  arXiv  Detail & Related papers  (2025-02-07T06:19:55Z)
• GAMA: Generative Agents for Multi-Agent Autoformalization [3.5083201638203154]
  We present a framework that enables the autoformalization of interaction scenarios using agents augmented by large language models (LLMs) The agents translate natural language descriptions of interactions into executable logic programs that define the rules of each game. A tournament simulation then tests the functionality of the generated game rules and strategies.
  arXiv  Detail & Related papers  (2024-12-11T22:37:45Z)
• Policy Learning with a Language Bottleneck [65.99843627646018]
  We introduce Policy Learning with a Language Bottleneck (PLLB), a framework enabling AI agents to generate linguistic rules. PLLBB alternates between a *rule generation* step guided by language models, and an *update* step where agents learn new policies guided by rules. We show thatPLLB agents are able to learn more interpretable and generalizable behaviors, but can also share the learned rules with human users.
  arXiv  Detail & Related papers  (2024-05-07T08:40:21Z)
• Steering Language Models with Game-Theoretic Solvers [43.023261136434876]
  We introduce a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large language models (LLMs)<n>Specifically, by modelling the players, strategies and payoffs in a "game" of dialogue, we create a binding from natural language interactions to the conventional symbolic logic of game theory.<n>We focus on three domains that require different negotiation strategies: scheduling meetings, trading fruit and debate, and evaluate an LLM's generated language when guided by solvers.
  arXiv  Detail & Related papers  (2024-01-24T22:22:00Z)
• MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration [98.18244218156492]
  Large Language Models (LLMs) have significantly advanced natural language processing.<n>As their applications expand into multi-agent environments, there arises a need for a comprehensive evaluation framework.<n>This work introduces a novel competition-based benchmark framework to assess LLMs within multi-agent settings.
  arXiv  Detail & Related papers  (2023-11-14T21:46:27Z)
• ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents [77.34720446306419]
  Alympics is a systematic simulation framework utilizing Large Language Model (LLM) agents for game theory research. Alympics creates a versatile platform for studying complex game theory problems.
  arXiv  Detail & Related papers  (2023-11-06T16:03:46Z)
• Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations [59.056367787688146]
  This paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs. We construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages. By utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages.
  arXiv  Detail & Related papers  (2023-10-31T08:09:20Z)
• Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game [40.438765131992525]
  We develop strategic language agents that generate flexible language actions and possess strong decision-making abilities. To mitigate the intrinsic bias in language actions, our agents use an LLM to perform deductive reasoning and generate a diverse set of action candidates. Experiments show that our agents overcome the intrinsic bias and outperform existing LLM-based agents in the Werewolf game.
  arXiv  Detail & Related papers  (2023-10-29T09:02:57Z)
• Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation [52.930183136111864]
  We propose using scorable negotiation to evaluate Large Language Models (LLMs) To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities. We provide procedures to create new games and increase games' difficulty to have an evolving benchmark.
  arXiv  Detail & Related papers  (2023-09-29T13:33:06Z)
• Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation [24.05034588588407]
  This paper addresses a key aspect in the design of such agents: Predicting human decision in off-policy evaluation. We collected a dataset of 87K decisions from humans playing a repeated decision-making game with artificial agents. Our approach involves training a model on human interactions with one agents subset to predict decisions when interacting with another.
  arXiv  Detail & Related papers  (2023-05-17T16:38:11Z)
• Computational Language Acquisition with Theory of Mind [84.2267302901888]
  We build language-learning agents equipped with Theory of Mind (ToM) and measure its effects on the learning process. We find that training speakers with a highly weighted ToM listener component leads to performance gains in our image referential game setting.
  arXiv  Detail & Related papers  (2023-03-02T18:59:46Z)
• Keep CALM and Explore: Language Models for Action Generation in Text-based Games [27.00685301984832]
  We propose the Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. We combine CALM with a reinforcement learning agent which re-ranks the generated action candidates to maximize in-game rewards.
  arXiv  Detail & Related papers  (2020-10-06T17:36:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.