MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind

• URL: http://arxiv.org/abs/2504.18039v1
• Date: Fri, 25 Apr 2025 03:12:43 GMT
• Title: MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind
• Authors: Zheng Zhang, Nuoqian Xiao, Qi Chai, Deheng Ye, Hao Wang,
• Abstract summary: MultiMind is the first framework integrating multimodal information into social deduction agents.<n>It processes facial expressions and vocal tones alongside verbal content, while employing a Theory of Mind (ToM) model.<n>By combining this ToM model with Monte Carlo Tree Search (MCTS), our agent identifies communication strategies that minimize suspicion directed at itself.
• Score: 17.2922544295112
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Model (LLM) agents have demonstrated impressive capabilities in social deduction games (SDGs) like Werewolf, where strategic reasoning and social deception are essential. However, current approaches remain limited to textual information, ignoring crucial multimodal cues such as facial expressions and tone of voice that humans naturally use to communicate. Moreover, existing SDG agents primarily focus on inferring other players' identities without modeling how others perceive themselves or fellow players. To address these limitations, we use One Night Ultimate Werewolf (ONUW) as a testbed and present MultiMind, the first framework integrating multimodal information into SDG agents. MultiMind processes facial expressions and vocal tones alongside verbal content, while employing a Theory of Mind (ToM) model to represent each player's suspicion levels toward others. By combining this ToM model with Monte Carlo Tree Search (MCTS), our agent identifies communication strategies that minimize suspicion directed at itself. Through comprehensive evaluation in both agent-versus-agent simulations and studies with human players, we demonstrate MultiMind's superior performance in gameplay. Our work presents a significant advancement toward LLM agents capable of human-like social reasoning across multimodal domains.

Related papers

• Unified Mind Model: Reimagining Autonomous Agents in the LLM Era [1.3812010983144802]
  Large language models (LLMs) have recently demonstrated remarkable capabilities across domains, tasks, and languages. We propose a novel theoretical cognitive architecture, the Unified Mind Model (UMM), which offers guidance to facilitate the rapid creation of autonomous agents.
  arXiv  Detail & Related papers  (2025-03-05T12:49:44Z)
• Towards Anthropomorphic Conversational AI Part I: A Practical Framework [49.62013440962072]
  We introduce a multi- module framework designed to replicate the key aspects of human intelligence involved in conversations.<n>In the second stage of our approach, these conversational data, after filtering and labeling, can serve as training and testing data for reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-28T03:18:39Z)
• MageBench: Bridging Large Multimodal Models to Agents [90.59091431806793]
  LMMs have shown impressive visual understanding capabilities, with the potential to be applied in agents.<n>Existing benchmarks mostly assess their reasoning abilities in language part.<n>MageBench is a reasoning capability oriented multimodal agent benchmark.
  arXiv  Detail & Related papers  (2024-12-05T17:08:19Z)
• MuMA-ToM: Multi-modal Multi-Agent Theory of Mind [10.079620078670589]
  We introduce MuMA-ToM, a Multi-modal Multi-Agent Theory of Mind benchmark.<n>We provide video and text descriptions of people's multi-modal behavior in realistic household environments.<n>We then ask questions about people's goals, beliefs, and beliefs about others' goals.
  arXiv  Detail & Related papers  (2024-08-22T17:41:45Z)
• Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Models [52.894048516550065]
  We develop a pipeline for multimodal ToM reasoning using video and text. We also enable explicit ToM reasoning by retrieving key frames for answering a ToM question.
  arXiv  Detail & Related papers  (2024-06-19T18:24:31Z)
• MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs [12.987019067098414]
  We propose a multi-agent Minecraft simulator, MineLand, that bridges this gap by introducing three key features: large-scale scalability, limited multimodal senses, and physical needs. Our simulator supports 64 or more agents. Agents have limited visual, auditory, and environmental awareness, forcing them to actively communicate and collaborate to fulfill physical needs like food and resources. Our experiments demonstrate that the simulator, the corresponding benchmark, and the AI agent framework contribute to more ecological and nuanced collective behavior.
  arXiv  Detail & Related papers  (2024-03-28T09:53:41Z)
• MMToM-QA: Multimodal Theory of Mind Question Answering [80.87550820953236]
  Theory of Mind (ToM) is an essential ingredient for developing machines with human-level social intelligence. Recent machine learning models, particularly large language models, seem to show some aspects of ToM understanding. Human ToM, on the other hand, is more than video or text understanding. People can flexibly reason about another person's mind based on conceptual representations extracted from any available data.
  arXiv  Detail & Related papers  (2024-01-16T18:59:24Z)
• SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems [53.94772445896213]
  Large Language Model (LLM)-based multi-agent systems have demonstrated promising performance in simulating human society. We propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication.
  arXiv  Detail & Related papers  (2024-01-08T15:01:08Z)
• ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents [77.34720446306419]
  Alympics is a systematic simulation framework utilizing Large Language Model (LLM) agents for game theory research. Alympics creates a versatile platform for studying complex game theory problems.
  arXiv  Detail & Related papers  (2023-11-06T16:03:46Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.