Book your room in the Turing Hotel! A symmetric and distributed Turing Test with multiple AIs and humans

• URL: http://arxiv.org/abs/2603.18981v1
• Date: Thu, 19 Mar 2026 14:44:47 GMT
• Title: Book your room in the Turing Hotel! A symmetric and distributed Turing Test with multiple AIs and humans
• Abstract summary: We report our experience with TuringHotel'', a novel extension of the Turing Test based on interactions within mixed communities of Large Language Models (LLMs) and human participants.<n>The classical one-to-one interaction of the Turing Test is reinterpreted in a group setting, where both human and artificial agents engage in time-bounded discussions and, interestingly, are both judges and respondents.<n>Results of our experimentation involving 17 human participants and 19 LLMs revealed that current models are still sometimes confused as humans.
• Score: 11.294156739799654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we report our experience with ``TuringHotel'', a novel extension of the Turing Test based on interactions within mixed communities of Large Language Models (LLMs) and human participants. The classical one-to-one interaction of the Turing Test is reinterpreted in a group setting, where both human and artificial agents engage in time-bounded discussions and, interestingly, are both judges and respondents. This community is instantiated in the novel platform UNaIVERSE (https://unaiverse.io), creating a ``World'' which defines the roles and interaction dynamics, facilitated by the platform's built-in programming tools. All communication occurs over an authenticated peer-to-peer network, ensuring that no third parties can access the exchange. The platform also provides a unified interface for humans, accessible via both mobile devices and laptops, that was a key component of the experience in this paper. Results of our experimentation involving 17 human participants and 19 LLMs revealed that current models are still sometimes confused as humans. Interestingly, there are several unexpected mistakes, suggesting that human fingerprints are still identifiable but not fully unambiguous, despite the high-quality language skills of artificial participants. We argue that this is the first experiment conducted in such a distributed setting, and that similar initiatives could be of national interest to support ongoing experiments and competitions aimed at monitoring the evolution of large language models over time.

Related papers

• Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue [1.1305136905804842]
  We present Inverse Turing Bench, a benchmark that evaluates models on their ability to differentiate humans and AI in multi-turn text.<n>The benchmark provides a collection of paired dialogue transcripts, wherein one dialogue is between two humans and the other is between a human and an AI.<n>We evaluated a preliminary set of models against this benchmark, and found that GPTZero, Claude Opus-4.6, and GPT-5.5 achieve the highest accuracy.
  arXiv  Detail & Related papers  (2026-06-20T02:47:56Z)
• CoGrid & the Multi-User Gymnasium: A Framework for Multi-Agent Experimentation [0.17188280334580197]
  CoGrid is a multi-agent grid-based simulation library with dual NumPy and JAX backends.<n>Multi-User Gymnasium translates such simulation environments directly into interactive web-based experiments.<n> MUG supports interactions with arbitrary numbers of humans and AI, utilizing either server-authoritative or peer-to-peer networking with rollback netcode to account for latency.
  arXiv  Detail & Related papers  (2026-04-16T14:10:38Z)
• Humanlike Multi-user Agent (HUMA): Designing a Deceptively Human AI Facilitator for Group Chats [0.0]
  We present the Humanlike Multi-user Agent (HUMA), an AI facilitator that participates in multi-party conversations using human-like strategies and timing.<n>Our results suggest that, in natural group chat settings, an AI facilitator can match human quality while remaining difficult to identify as nonhuman.
  arXiv  Detail & Related papers  (2025-11-21T15:34:42Z)
• Towards Anthropomorphic Conversational AI Part I: A Practical Framework [49.62013440962072]
  We introduce a multi- module framework designed to replicate the key aspects of human intelligence involved in conversations.<n>In the second stage of our approach, these conversational data, after filtering and labeling, can serve as training and testing data for reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-28T03:18:39Z)
• Multi-Turn Human-LLM Interaction Through the Lens of a Two-Way Intelligibility Protocol [1.5711521670164208]
  We investigate a more structured approach based on an abstract protocol for interaction between agents.<n>The protocol is motivated by a notion of "two-way intelligibility" and is modelled by a pair of communicating finite-state machines.<n>The results provide evidence in support of the protocol's capability of capturing one- and two-way intelligibility in human-LLM interaction.
  arXiv  Detail & Related papers  (2024-10-27T21:20:18Z)
• X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents [56.64615470513102]
  The Turing test examines whether AIs exhibit human-like behaviour in natural language conversations.<n>Traditional setting limits each participant to one message at a time and requires constant human participation.<n>This paper proposes textbftextscX-Turing, which enhances the original test with a textitburst dialogue pattern.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO [55.25989137825992]
  We introduce ECHO, an evaluative framework inspired by the Turing test. This framework engages the acquaintances of the target individuals to distinguish between human and machine-generated responses. We evaluate three role-playing LLMs using ECHO, with GPT-3.5 and GPT-4 serving as foundational models.
  arXiv  Detail & Related papers  (2024-04-22T08:00:51Z)
• Exploring Large Language Models to Facilitate Variable Autonomy for Human-Robot Teaming [4.779196219827508]
  We introduce a novel framework for a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with robot agents through natural language, each powered by individual GPT cores. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a multi-robot environment.
  arXiv  Detail & Related papers  (2023-12-12T12:26:48Z)
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy [70.45420918526926]
  We present LILAC, a framework for incorporating and adapting to natural language corrections online during execution. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot. We show that our corrections-aware approach obtains higher task completion rates, and is subjectively preferred by users.
  arXiv  Detail & Related papers  (2023-01-06T15:03:27Z)
• Evaluating Human-Language Model Interaction [79.33022878034627]
  We develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems. We design five tasks to cover different forms of interaction: social dialogue, question answering, crossword puzzles, summarization, and metaphor generation. We find that better non-interactive performance does not always translate to better human-LM interaction.
  arXiv  Detail & Related papers  (2022-12-19T18:59:45Z)
• Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks [83.37025218216888]
  We propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations. The robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications. Results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.
  arXiv  Detail & Related papers  (2020-07-24T23:35:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.