Deliberate Lab: A Platform for Real-Time Human-AI Social Experiments

• URL: http://arxiv.org/abs/2510.13011v1
• Date: Tue, 14 Oct 2025 22:02:24 GMT
• Title: Deliberate Lab: A Platform for Real-Time Human-AI Social Experiments
• Authors: Crystal Qian, Vivian Tsai, Michael Behr, Nada Hussein, Léo Laugier, Nithum Thain, Lucas Dixon,
• Abstract summary: Deliberate Lab is an open-source platform for large-scale, real-time behavioral experiments.<n>It supports both human participants and large language model (LLM)-based agents.
• Score: 9.689197691319741
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social and behavioral scientists increasingly aim to study how humans interact, collaborate, and make decisions alongside artificial intelligence. However, the experimental infrastructure for such work remains underdeveloped: (1) few platforms support real-time, multi-party studies at scale; (2) most deployments require bespoke engineering, limiting replicability and accessibility, and (3) existing tools do not treat AI agents as first-class participants. We present Deliberate Lab, an open-source platform for large-scale, real-time behavioral experiments that supports both human participants and large language model (LLM)-based agents. We report on a 12-month public deployment of the platform (N=88 experimenters, N=9195 experiment participants), analyzing usage patterns and workflows. Case studies and usage scenarios are aggregated from platform users, complemented by in-depth interviews with select experimenters. By lowering technical barriers and standardizing support for hybrid human-AI experimentation, Deliberate Lab expands the methodological repertoire for studying collective decision-making and human-centered AI.

Related papers

• SelfAI: Building a Self-Training AI System with LLM Agents [79.10991818561907]
  SelfAI is a general multi-agent platform that combines a User Agent for translating high-level research objectives into standardized experimental configurations.<n>An Experiment Manager orchestrates parallel, fault-tolerant training across heterogeneous hardware while maintaining a structured knowledge base for continuous feedback.<n>Across regression, computer vision, scientific computing, medical imaging, and drug discovery benchmarks, SelfAI consistently achieves strong performance and reduces redundant trials.
  arXiv  Detail & Related papers  (2025-11-29T09:18:39Z)
• Measuring skill-based uplift from AI in a real biological laboratory [0.0]
  We report the results of a pilot study that attempted to empirically measure the magnitude of emphskills-based uplift caused by access to an AI reasoning model.<n>We discuss these results in the context of future studies of the evolving relationship between AI and global biosecurity.
  arXiv  Detail & Related papers  (2025-10-29T16:34:57Z)
• LabOS: The AI-XR Co-Scientist That Sees and Works With Humans [51.025615465050635]
  LabOS represents the first AI co-scientist that unites computational reasoning with physical experimentation.<n>By connecting multi-model AI agents, smart glasses, and human-AI collaboration, LabOS allows AI to see what scientists see, understand experimental context, and assist in real-time execution.
  arXiv  Detail & Related papers  (2025-10-16T16:36:22Z)
• Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration [36.471054708769415]
  Large language model (LLM) agents open new opportunities for human-LLM-agent collaboration.<n>It remains unclear whether principles of computer-mediated collaboration established in HCI and CSCW persist, change, or fail when humans collaborate with LLM agents.
  arXiv  Detail & Related papers  (2025-09-22T16:47:08Z)
• An AI-native experimental laboratory for autonomous biomolecular engineering [12.382004681010915]
  We present an AI-native autonomous laboratory, targeting highly complex scientific experiments for applications like autonomous biomolecular engineering.<n>This system autonomously manages instrumentation, formulates experiment-specific procedures and optimizations, and concurrently serves multiple user requests.<n>It also enables applications in fields such as disease diagnostics, drug development, and information storage.
  arXiv  Detail & Related papers  (2025-07-03T07:21:19Z)
• Epitome: Pioneering an Experimental Platform for AI-Social Science Integration [29.742040167436638]
  We introduce Epitome, the world's first open experimental platform dedicated to the deep integration of artificial intelligence and social science.<n>Epitome focuses on the interactive impacts of AI on individuals, organizations, and society during its real-world deployment.<n>With its canvas-style, user-friendly interface, Epitome enables researchers to easily design and run complex experimental scenarios.
  arXiv  Detail & Related papers  (2025-06-30T09:06:16Z)
• Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI [98.19195693735487]
  We propose the paradigm of Intelligent Science Laboratories (ISLs)<n>ISLs are a multi-layered, closed-loop framework that deeply integrates cognitive and embodied intelligence.<n>We argue that such systems are essential for overcoming the current limitations of scientific discovery.
  arXiv  Detail & Related papers  (2025-06-24T13:31:44Z)
• Human-Machine Teaming for UAVs: An Experimentation Platform [6.809734620480709]
  We present the Cogment human-machine teaming experimentation platform. It implements human-machine teaming (HMT) use cases that can involve learning AI agents, static AI agents, and humans. We hope to facilitate further research on human-machine teaming in critical systems and defense environments.
  arXiv  Detail & Related papers  (2023-12-18T21:35:02Z)
• User Behavior Simulation with Large Language Model based Agents [116.74368915420065]
  We propose an LLM-based agent framework and design a sandbox environment to simulate real user behaviors. Based on extensive experiments, we find that the simulated behaviors of our method are very close to the ones of real humans.
  arXiv  Detail & Related papers  (2023-06-05T02:58:35Z)
• BO-Muse: A human expert and AI teaming framework for accelerated experimental design [58.61002520273518]
  Our algorithm lets the human expert take the lead in the experimental process. We show that our algorithm converges sub-linearly, at a rate faster than the AI or human alone.
  arXiv  Detail & Related papers  (2023-03-03T02:56:05Z)
• DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation [91.3755431537592]
  This work presents DIAMBRA Arena, a new platform for reinforcement learning research and experimentation. It features a collection of high-quality environments exposing a Python API fully compliant with OpenAI Gym standard. They are episodic tasks with discrete actions and observations composed by raw pixels plus additional numerical values.
  arXiv  Detail & Related papers  (2022-10-19T14:39:10Z)
• Taxonomy of A Decision Support System for Adaptive Experimental Design in Field Robotics [19.474298062145003]
  We propose a Decision Support System (DSS) to amplify the human's decision-making abilities and enable principled decision-making in field experiments. We construct and present our taxonomy using examples and trends from DSS literature, including works involving artificial intelligence and Intelligent DSSs.
  arXiv  Detail & Related papers  (2022-10-15T23:28:30Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.