Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolf
- URL: http://arxiv.org/abs/2405.19946v1
- Date: Thu, 30 May 2024 11:07:06 GMT
- Title: Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolf
- Authors: Xuanfa Jin, Ziyan Wang, Yali Du, Meng Fang, Haifeng Zhang, Jun Wang,
- Abstract summary: As a variant of the famous communication game Werewolf, One Night Ultimate Werewolf (ONUW) requires players to develop strategic discussion policies.
We propose an RL-instructed language agent framework, where a discussion policy trained by reinforcement learning (RL) is employed to determine appropriate discussion tactics to adopt.
- Score: 28.57358844115881
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Communication is a fundamental aspect of human society, facilitating the exchange of information and beliefs among people. Despite the advancements in large language models (LLMs), recent agents built with these often neglect the control over discussion tactics, which are essential in communication scenarios and games. As a variant of the famous communication game Werewolf, One Night Ultimate Werewolf (ONUW) requires players to develop strategic discussion policies due to the potential role changes that increase the uncertainty and complexity of the game. In this work, we first present the existence of the Perfect Bayesian Equilibria (PBEs) in two scenarios of the ONUW game: one with discussion and one without. The results showcase that the discussion greatly changes players' utilities by affecting their beliefs, emphasizing the significance of discussion tactics. Based on the insights obtained from the analyses, we propose an RL-instructed language agent framework, where a discussion policy trained by reinforcement learning (RL) is employed to determine appropriate discussion tactics to adopt. Our experimental results on several ONUW game settings demonstrate the effectiveness and generalizability of our proposed framework.
Related papers
- Verbalized Bayesian Persuasion [54.55974023595722]
Information design (ID) explores how a sender influence the optimal behavior of receivers to achieve specific objectives.
This work proposes a verbalized framework in Bayesian persuasion (BP), which extends classic BP to real-world games involving human dialogues for the first time.
Numerical experiments in dialogue scenarios, such as recommendation letters, courtroom interactions, and law enforcement, validate that our framework can both reproduce theoretical results in classic BP and discover effective persuasion strategies.
arXiv Detail & Related papers (2025-02-03T18:20:10Z) - Enhancing Dialogue Generation in Werewolf Game Through Situation Analysis and Persuasion Strategies [1.7725414095035827]
This paper introduces a LLM-based Werewolf Game AI, where each role is supported by situation analysis to aid response generation.
Various persuasion strategies are employed to effectively persuade other players to align with its actions.
arXiv Detail & Related papers (2024-08-29T14:49:13Z) - Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent.
We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents.
Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
arXiv Detail & Related papers (2024-06-16T15:24:50Z) - Helmsman of the Masses? Evaluate the Opinion Leadership of Large Language Models in the Werewolf Game [1.4565642534804486]
We employ the Werewolf game as a simulation platform to assess the opinion leadership of large language models (LLMs)
The game includes the role of the Sheriff, tasked with summarizing arguments and recommending decision options.
We devise two novel metrics based on the critical characteristics of opinion leaders.
arXiv Detail & Related papers (2024-04-02T02:46:18Z) - CivRealm: A Learning and Reasoning Odyssey in Civilization for
Decision-Making Agents [63.79739920174535]
We introduce CivRealm, an environment inspired by the Civilization game.
CivRealm stands as a unique learning and reasoning challenge for decision-making agents.
arXiv Detail & Related papers (2024-01-19T09:14:11Z) - Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf [19.39740531672788]
We propose a tuning-free framework to engage large language models in communication games.
An empirical study on the representative and widely-studied communication game, Werewolf'', demonstrates that our framework can effectively play Werewolf game without tuning the parameters of the LLMs.
arXiv Detail & Related papers (2023-09-09T01:56:40Z) - Improving Language Model Negotiation with Self-Play and In-Context
Learning from AI Feedback [97.54519989641388]
We study whether multiple large language models (LLMs) can autonomously improve each other in a negotiation game by playing, reflecting, and criticizing.
Only a subset of the language models we consider can self-play and improve the deal price from AI feedback.
arXiv Detail & Related papers (2023-05-17T11:55:32Z) - Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion
Behaviors in Social Deduction Games [45.55448048482881]
We introduce the first multimodal dataset for modeling persuasion behaviors.
Our dataset includes 199 dialogue transcriptions and videos, 26,647 utterance level annotations of persuasion strategy, and game level annotations of deduction game outcomes.
arXiv Detail & Related papers (2022-12-16T04:52:53Z) - A Novel Weighted Ensemble Learning Based Agent for the Werewolf Game [0.0]
Werewolf is a popular party game throughout the world, and research on its significance has progressed in recent years.
In this research, we generated a sophisticated agent to play the Werewolf game using a complex weighted ensemble learning approach.
arXiv Detail & Related papers (2022-05-19T19:19:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.