Conversation Games and a Strategic View of the Turing Test

• URL: http://arxiv.org/abs/2501.18455v1
• Date: Thu, 30 Jan 2025 16:08:37 GMT
• Title: Conversation Games and a Strategic View of the Turing Test
• Abstract summary: We focus on a subset of the games, called verdict games.<n>In a verdict game, two players alternate to contribute to a conversation, which is evaluated at each stage by a non-strategic judge.<n>We show the practical relevance of the proposed concepts by simulation experiments, and show that a strategic agent outperforms a naive agent by a high margin.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although many game-theoretic models replicate real interactions that often rely on natural language, explicit study of games where language is central to strategic interaction remains limited. This paper introduces the \emph{conversation game}, a multi-stage, extensive-form game based on linguistic strategic interaction. We focus on a subset of the games, called verdict games. In a verdict game, two players alternate to contribute to a conversation, which is evaluated at each stage by a non-strategic judge who may render a conclusive binary verdict, or a decision to continue the dialogue. The game ends once a limit is reached or a verdict is given. We show many familiar processes, such as interrogation or a court process fall under this category. We also, show that the Turing test is an instance of verdict game, and discuss the significance of a strategic view of the Turing test in the age of advanced AI deception. We show the practical relevance of the proposed concepts by simulation experiments, and show that a strategic agent outperforms a naive agent by a high margin.

Related papers

• Investigating Knowledge Transfer Across Interactive Dialogue Games [73.44278078966448]
  We investigate how knowledge transfers across different dialogue games.<n>We study transferability by finetuning LLM models on games from the clembench suite.<n>We find that some games benefit more from transfer than finetuning, and that the visuospatial family (e.g., exploration games) transfers best.
  arXiv  Detail & Related papers  (2026-08-25T01:47:31Z)
• RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue [2.7606655162305476]
  We present RogueAI, an interactive webapp that operationalizes this test as a one-on-two interrogation game.<n>The player's task is to identify the deceptive agent and "shut it off" before a turn budget is exhausted.<n>We introduce AutoRogueAI, a procedural extension in which players co-design a custom scenario with a narrator agent that secretly chooses its own deception strategy.
  arXiv  Detail & Related papers  (2026-06-11T13:07:02Z)
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
  We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
  arXiv  Detail & Related papers  (2026-05-28T07:33:47Z)
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents [76.60994803070436]
  GameWorld is a benchmark for evaluation of Multimodal Large Language Model (MLLM) game agents in browser environments.<n>Two game agent interfaces are studied: (i) computer-use agents that directly emit keyboard and mouse controls, and (ii) generalist multimodal agents that act in a semantic action space.<n>Results across 18 model-interface pairs suggest that even the best performing agent is far from achieving human capabilities on video games.
  arXiv  Detail & Related papers  (2026-04-08T17:49:03Z)
• People use fast, flat goal-directed simulation to reason about novel problems [68.55490343866545]
  We show that people are systematic and adaptively rational in how they play a game for the first time.<n>We explain these capacities via a computational cognitive model that we call the "Intuitive Gamer"<n>Our work offers new insights into how people rapidly evaluate, act, and make suggestions when encountering novel problems.
  arXiv  Detail & Related papers  (2025-10-13T15:12:08Z)
• Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies [54.08697738311866]
  Social deduction games like Werewolf combine language, reasoning, and strategy.<n>We curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants.<n>We propose a novel strategy-alignment evaluation that leverages the winning faction's strategies as ground truth in two stages.
  arXiv  Detail & Related papers  (2025-10-13T13:33:30Z)
• Evaluating Language Models' Evaluations of Games [65.49017696754825]
  We advocate for a new paradigm that assesses AI systems' evaluation of games.<n>We leverage a large-scale dataset of over $100$ novel board games and over 450 human judgments.<n>Our results show that reasoning models are generally more aligned to people in their evaluations of games than non-reasoning language models.
  arXiv  Detail & Related papers  (2025-10-13T02:45:37Z)
• Dynamic Coalition Structure Detection in Natural Language-based Interactions [29.093071108373717]
  In strategic multi-agent sequential interactions, detecting dynamic coalition structures is crucial for understanding how self-interested agents coordinate to influence outcomes. We propose a new method to predict dynamic multilateral coalition formation in Diplomacy, a strategic multi-agent game where agents negotiate coalitions using natural language.
  arXiv  Detail & Related papers  (2025-02-22T20:12:52Z)
• Self-Directed Turing Test for Large Language Models [56.64615470513102]
  The Turing test examines whether AIs can exhibit human-like behaviour in natural language conversations. Traditional Turing tests adopt a rigid dialogue format where each participant sends only one message each time. This paper proposes the Self-Directed Turing Test, which extends the original test with a burst dialogue format.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games [21.639516389561837]
  We introduce WellPlay, a reasoning dataset for multi-agent conversational inference in Murder Mystery Games (MMGs)<n>WellPlay comprises 1,482 inferential questions across 12 games, spanning objectives, reasoning, and relationship understanding.<n>We present PLAYER*, a novel framework for Large Language Model (LLM)-based agents in MMGs.
  arXiv  Detail & Related papers  (2024-04-26T19:07:30Z)
• Open-Ended Wargames with Large Language Models [3.2228025627337864]
  We introduce "Snow Globe," an LLM-powered multi-agent system for playing qualitative wargames. We describe its software architecture conceptually and release an open-source implementation alongside this publication. We discuss potential applications of the approach and how it fits into the broader wargaming ecosystem.
  arXiv  Detail & Related papers  (2024-04-17T14:54:58Z)
• Steering Language Models with Game-Theoretic Solvers [43.023261136434876]
  We introduce a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large language models (LLMs)<n>Specifically, by modelling the players, strategies and payoffs in a "game" of dialogue, we create a binding from natural language interactions to the conventional symbolic logic of game theory.<n>We focus on three domains that require different negotiation strategies: scheduling meetings, trading fruit and debate, and evaluate an LLM's generated language when guided by solvers.
  arXiv  Detail & Related papers  (2024-01-24T22:22:00Z)
• Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games [45.55448048482881]
  We introduce the first multimodal dataset for modeling persuasion behaviors. Our dataset includes 199 dialogue transcriptions and videos, 26,647 utterance level annotations of persuasion strategy, and game level annotations of deduction game outcomes.
  arXiv  Detail & Related papers  (2022-12-16T04:52:53Z)
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games [64.11746320061965]
  We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language. We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
  arXiv  Detail & Related papers  (2020-10-22T12:40:22Z)
• Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning [94.50608198582636]
  Interactive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. We take a novel perspective of IF game solving and re-formulate it as Multi-Passage Reading (MPRC) tasks.
  arXiv  Detail & Related papers  (2020-10-05T23:09:20Z)
• Learning to Play Sequential Games versus Unknown Opponents [93.8672371143881]
  We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response.
  arXiv  Detail & Related papers  (2020-07-10T09:33:05Z)
• Predicting Strategic Behavior from Free Text [38.506665373140876]
  We study the connection between messaging and action in an economic context, modeled as a game. We introduce the study of an individual's action prediction in a one-shot game based on free text he/she provides. We employ transductive learning to predict actions taken by these individuals in one-shot games based on these attributes.
  arXiv  Detail & Related papers  (2020-04-06T20:05:30Z)
• Efficient exploration of zero-sum stochastic games [83.28949556413717]
  We investigate the increasingly important and common game-solving setting where we do not have an explicit description of the game but only oracle access to it through gameplay. During a limited-duration learning phase, the algorithm can control the actions of both players in order to try to learn the game and how to play it well. Our motivation is to quickly learn strategies that have low exploitability in situations where evaluating the payoffs of a queried strategy profile is costly.
  arXiv  Detail & Related papers  (2020-02-24T20:30:38Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.