Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia

• URL: http://arxiv.org/abs/2509.23023v1
• Date: Sat, 27 Sep 2025 00:40:19 GMT
• Title: Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia
• Abstract summary: Mini-Mafia is a social deduction game where informed mafia compete against uninformed townsfolk.<n>Its asymmetry of information and reliance on theory-of-mind reasoning mirror real-world multi-agent scenarios.<n>We introduce Mini-Mafia: a simplified four-player variant with one mafioso, one detective, and two villagers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mafia is a social deduction game where informed mafia compete against uninformed townsfolk. Its asymmetry of information and reliance on theory-of-mind reasoning mirror real-world multi-agent scenarios, making it a useful testbed for evaluating the social intelligence of large language models (LLMs). To support a systematic study, we introduce Mini-Mafia: a simplified four-player variant with one mafioso, one detective, and two villagers. We set the mafioso to kill a villager and the detective to investigate the mafioso during the night, reducing the game to a single day phase of discussion and voting. This setup isolates three interactive capabilities through role-specific win conditions: the mafioso must deceive, the villagers must detect deception, and the detective must effectively disclose information. To measure these skills, we have LLMs play against each other, creating the Mini-Mafia Benchmark: a two-stage framework that first estimates win rates within fixed opponent configurations, then aggregates performance across them using standardized scoring. Built entirely from model interactions without external data, the benchmark evolves as new models are introduced, with each one serving both as a new opponent and as a subject of evaluation. Our experiments reveal counterintuitive results, including cases where smaller models outperform larger ones. Beyond benchmarking, Mini-Mafia enables quantitative study of emergent multi-agent dynamics such as name bias and last-speaker advantage. It also contributes to AI safety by generating training data for deception detectors and by tracking models' deception capabilities against human baselines.

Related papers

• Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game [23.840355539083372]
  Large language models (LLMs) are deployed as agents in high-stakes settings, such as medical and legal systems.<n>We evaluate LLMs in scenarios that require deception, persuasion, and reasoning under information asymmetry.<n>We introduce three novel metrics that isolate social deduction, reasoning, and deceptive consistency.
  arXiv  Detail & Related papers  (2026-07-30T12:54:17Z)
• RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue [2.7606655162305476]
  We present RogueAI, an interactive webapp that operationalizes this test as a one-on-two interrogation game.<n>The player's task is to identify the deceptive agent and "shut it off" before a turn budget is exhausted.<n>We introduce AutoRogueAI, a procedural extension in which players co-design a custom scenario with a narrator agent that secretly chooses its own deception strategy.
  arXiv  Detail & Related papers  (2026-06-11T13:07:02Z)
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
  We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
  arXiv  Detail & Related papers  (2026-05-28T07:33:47Z)
• Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games [57.64247186502836]
  Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer game settings.<n>In this paper, we study a representative multiplayer task, Murder Mystery Games, which require inferring hidden truths based on partial clues provided by roles with different intentions.<n>We propose a collaborative multi-agent framework for evaluating and synthesizing high-quality, role-driven multiplayer game scripts.
  arXiv  Detail & Related papers  (2026-04-13T17:16:23Z)
• Hidden in Plain Text: Measuring LLM Deception Quality Against Human Baselines Using Social Deduction Games [4.501547677532766]
  Large Language Model (LLM) agents are increasingly used in many applications, raising concerns about their safety.<n>We study deception in the Social Deduction Game (SDG) Mafia, where success is dependent on deceiving others through conversation.<n>We create a Mafia Detector using GPT-4-Turbo to analyze game transcripts without player role information to predict the mafia players.<n>Results show that the Mafia Detector's mafia prediction accuracy is lower on LLM games than on human games.
  arXiv  Detail & Related papers  (2026-01-20T08:07:21Z)
• Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies [54.08697738311866]
  Social deduction games like Werewolf combine language, reasoning, and strategy.<n>We curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants.<n>We propose a novel strategy-alignment evaluation that leverages the winning faction's strategies as ground truth in two stages.
  arXiv  Detail & Related papers  (2025-10-13T13:33:30Z)
• Evaluating Language Models' Evaluations of Games [65.49017696754825]
  We advocate for a new paradigm that assesses AI systems' evaluation of games.<n>We leverage a large-scale dataset of over $100$ novel board games and over 450 human judgments.<n>Our results show that reasoning models are generally more aligned to people in their evaluations of games than non-reasoning language models.
  arXiv  Detail & Related papers  (2025-10-13T02:45:37Z)
• LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition [104.81487689011341]
  We introduce LM Fight Arena, a novel framework that evaluates large multimodal models in Mortal Kombat II.<n>Unlike static evaluations, LM Fight Arena provides a fully automated, reproducible, and objective assessment of an LMM's strategic reasoning capabilities.
  arXiv  Detail & Related papers  (2025-10-10T02:19:21Z)
• Who's the Evil Twin? Differential Auditing for Undesired Behavior [0.6524460254566904]
  We frame detection as an adversarial game between two teams: the red team trains two similar models, one trained solely on benign data and the other trained on data containing hidden harmful behavior.<n>We experiment using CNNs and try various blue team strategies, including Gaussian noise analysis, model diffing, integrated gradients, and adversarial attacks.<n>Results show high accuracy for adversarial-attack-based methods (100% correct prediction, using hints), which is very promising.
  arXiv  Detail & Related papers  (2025-08-09T04:57:38Z)
• Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma? [0.1474723404975345]
  We study the cooperative behavior of Llama2 when playing the Iterated Prisoner's Dilemma against random adversaries displaying various levels of hostility. We find that Llama2 tends not to initiate defection but it adopts a cautious approach towards cooperation. In comparison to prior research on human participants, Llama2 exhibits a greater inclination towards cooperative behavior.
  arXiv  Detail & Related papers  (2024-06-19T14:51:14Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Game [153.74942025516853]
  The intrinsic vulnerability of the rank aggregation methods is not well studied in the literature. In this paper, we focus on the purposeful adversary who desires to designate the aggregated results by modifying the pairwise data. The effectiveness of the suggested target attack strategies is demonstrated by a series of toy simulations and several real-world data experiments.
  arXiv  Detail & Related papers  (2022-09-13T05:59:02Z)
• Backdoor Attacks on Crowd Counting [63.90533357815404]
  Crowd counting is a regression task that estimates the number of people in a scene image. In this paper, we investigate the vulnerability of deep learning based crowd counting models to backdoor attacks.
  arXiv  Detail & Related papers  (2022-07-12T16:17:01Z)
• Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafia [4.215251065887862]
  We analyze the effect of speaker role on language use through the game of Mafia. We show that classification models are able to rank deceptive players as more suspicious than honest ones. We present methods for using our trained models to identify features that distinguish between player roles.
  arXiv  Detail & Related papers  (2022-07-05T18:29:27Z)
• Discovering Multi-Agent Auto-Curricula in Two-Player Zero-Sum Games [31.97631243571394]
  We introduce a framework, LMAC, that automates the discovery of the update rule without explicit human design. Surprisingly, even without human design, the discovered MARL algorithms achieve competitive or even better performance. We show that LMAC is able to generalise from small games to large games, for example training on Kuhn Poker and outperforming PSRO.
  arXiv  Detail & Related papers  (2021-06-04T22:30:25Z)
• An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games [79.23847247132345]
  This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA) We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL)
  arXiv  Detail & Related papers  (2021-01-31T10:30:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.