The DeepSpeak-Agentic Dataset

• URL: http://arxiv.org/abs/2606.03686v1
• Date: Tue, 02 Jun 2026 14:10:18 GMT
• Title: The DeepSpeak-Agentic Dataset
• Abstract summary: We present DeepSpeak-Agentic, a dataset of videos comprising over 37 hours of semi-structured conversations between a human and an embodied AI agent.<n>We use this dataset to evaluate the automatic forensic identification of AI agents, study the nature of human-agent interactions, and provide a benchmark for future advances in the large-language models and AI-generated voices and faces that power embodied AI agents.
• Score: 11.151457846264185
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present DeepSpeak-Agentic, a dataset of videos comprising over 37 hours of semi-structured conversations between a human and an embodied AI agent. We use this dataset to evaluate the automatic forensic identification (audio, video, or text) of AI agents, study the nature of human-agent interactions, and provide a benchmark for future advances in the large-language models and AI-generated voices and faces that power embodied AI agents. We also contribute a scalable data-capture system that creates agents, automatically pairs them with human crowd workers, records audiovisual conversations across specified scenarios, and identifies and separates the human and agent in the combined stream.

Related papers

• ComBodied Agents: a New Paradigm of Human-Centric Agentic AI [121.60938707072069]
  We introduce Combodied Agents, a human-centered paradigm that perceives, models, predicts, and supports individual human-state trajectories over time.<n>We unify fragmented capabilities across personal assistants, health agents, AI companions, and adaptive human--AI systems into a closed loop.
  arXiv  Detail & Related papers  (2026-08-11T13:35:58Z)
• Adversarial Creation and Detection of AI-Generated Social Bot Content [48.21614389337061]
  Existing models for detecting AI-generated content often fail in the wild, primarily due to the lack of ground-truth data.<n>We address this gap through an adversarial methodology that models the impersonation of real social media users by malicious actors.<n>Our approach significantly outperforms existing models for content-based bot detection in real-world, out-of-distribution data.
  arXiv  Detail & Related papers  (2026-06-05T12:32:47Z)
• AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science [30.303936447998314]
  Recent developments in large language models (LLMs) and artificial intelligence (AI) agents have significantly automated data science workflow.<n>It remains unclear to what extent AI agents can match the performance of human experts on domain-specific data science tasks.<n>We introduce AgentDS, a benchmark and competition designed to evaluate both AI agents and human-AI collaboration performance in domain-specific data science.
  arXiv  Detail & Related papers  (2026-03-19T15:11:13Z)
• MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents [54.48066948369172]
  As embodied models become powerful, humans will collaborate with multiple embodied AI agents at their workplace or home in the future.<n>Existing challenges include effectively compressing and communicating high volumes of individual sensory inputs in the form of video.<n>We first formally define a novel problem of understanding multiple long-horizon egocentric videos simultaneously collected from embodied agents.
  arXiv  Detail & Related papers  (2026-03-10T15:48:35Z)
• Cloning a Conversational Voice AI Agent from Call\,Recording Datasets for Telesales [0.0]
  We present a methodology for cloning a conversational voice AI agent from a corpus of call recordings.<n>Our system listens to customers over the telephone, responds with a synthetic voice, and follows a structured playbook learned from top performing human agents.<n>The cloned agent is evaluated against human agents on a rubric of 22 criteria covering introduction, product communication, sales drive, objection handling, and closing.
  arXiv  Detail & Related papers  (2025-09-05T07:36:12Z)
• Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions [110.43343503158306]
  This paper embeds the manual-assisted task into a vision-language-action framework, where the assistant provides services to the instructor following egocentric vision and commands.<n>Under this setting, we accomplish InterVLA, the first large-scale human-object-human interaction dataset with 11.4 hours and 1.2M frames of multimodal data.<n>We establish novel benchmarks on egocentric human motion estimation, interaction synthesis, and interaction prediction with comprehensive analysis.
  arXiv  Detail & Related papers  (2025-08-06T17:46:23Z)
• Agentic Web: Weaving the Next Web with AI Agents [109.13815627467514]
  The emergence of AI agents powered by large language models (LLMs) marks a pivotal shift toward the Agentic Web.<n>In this paradigm, agents interact directly with one another to plan, coordinate, and execute complex tasks on behalf of users.<n>We present a structured framework for understanding and building the Agentic Web.
  arXiv  Detail & Related papers  (2025-07-28T17:58:12Z)
• A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions [51.96890647837277]
  Large Language Models (LLMs) have propelled conversational AI from traditional dialogue systems into sophisticated agents capable of autonomous actions, contextual awareness, and multi-turn interactions with users.<n>This survey paper presents a desideratum for next-generation Conversational Agents - what has been achieved, what challenges persist, and what must be done for more scalable systems that approach human-level intelligence.
  arXiv  Detail & Related papers  (2025-04-07T21:01:25Z)
• ConvoGen: Enhancing Conversational AI with Synthetic Data: A Multi-Agent Approach [0.0]
  We present ConvoGen, an innovative framework for generating synthetic conversational data using multi-agent systems.<n>The generated data has numerous applications, including training and evaluating conversational AI models.
  arXiv  Detail & Related papers  (2025-03-21T18:14:12Z)
• IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents [20.460482488872145]
  This paper addresses the challenges of developing interactive agents capable of understanding and executing grounded natural language instructions. We introduce a scalable data collection tool for gathering interactive grounded language instructions within a Minecraft-like environment. We present a Human-in-the-Loop interactive evaluation platform for qualitative analysis and comparison of agent performance.
  arXiv  Detail & Related papers  (2024-07-12T00:07:43Z)
• Training Conversational Agents with Generative Conversational Networks [74.9941330874663]
  We use Generative Conversational Networks to automatically generate data and train social conversational agents. We evaluate our approach on TopicalChat with automatic metrics and human evaluators, showing that with 10% of seed data it performs close to the baseline that uses 100% of the data.
  arXiv  Detail & Related papers  (2021-10-15T21:46:39Z)
• A Study on AI-FML Robotic Agent for Student Learning Behavior Ontology Construction [1.0183055506531897]
  The proposed AI-FML robotic agent is applied in English speaking and listening domain. There are three intelligent agents, including a perception agent, a computational agent, and a cognition agent in the AI-FML robotic agent. The experimental results show that the agents can be utilized in the human and machine co-learning model for the future education.
  arXiv  Detail & Related papers  (2020-06-18T01:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.