Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents

• URL: http://arxiv.org/abs/2609.13117v1
• Date: Fri, 11 Sep 2026 17:46:10 GMT
• Title: Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents
• Abstract summary: We introduce Duplex Cue, an evaluation of thisemphin adaptation in full voice agents.<n>In a single-model study using 300 human-confirmed unscripted English conversations, we compare recorded human responses generated while replaying listener's cues.<n>Speakers adapt in 68.2% of cases, compared with 32.4% for PersonaPlex.
• Score: 2.745780834143671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Full-duplex evaluation often emphasizes whether an agent keeps speaking or stops. That binary cannot express a third response humans use routinely: continuing to speak while incorporating what the listener just contributed. The contribution may be a missing word, a correction or a clarification. We introduce Duplex Cue, an evaluation of this \emph{in-turn adaptation} in full-duplex voice agents. Duplex Cue separates listener intent (backchannel, collaboration, or interruption) from speaker behavior: continuing unchanged, adapting within the turn, or yielding. Adaptation includes acknowledgment as well as content revision. In a single-model case study using 300 human-confirmed cues from unscripted English conversations, we compare recorded human responses with PersonaPlex continuations generated while replaying the listener's audio. We retain 208 pairs with the ongoing speaker active at cue onset and a scorable response in each condition. On the 66 collaborative pairs, recorded speakers adapt in 68.2\% of cases, compared with 34.8\% for PersonaPlex. The model otherwise continues unchanged (42.4\%) or yields (22.7\%). These findings show why evaluating natural voice interaction requires measuring how an agent responds to a listener's contribution as well as whether it keeps speaking.

Related papers

• Realtime-Venus: A full-duplex interaction system with asynchronous delegation [59.69745588222872]
  We present Realtime, Realtime Omni audio visual interaction and Realtime-Audio Spoken for spoken interaction.<n>Each model serves as complete conversational, integrating continuous perception, conversational control, and native speech generation.
  arXiv  Detail & Related papers  (2026-09-12T08:56:50Z)
• SteerDuplex: Steerable Duplex Speech Dialogue Models [47.277615720741046]
  Full spoken dialogue model fine-tuned on natural conversations and synthetic dialogues targeting following instruction, vocal delivery, reasoning, and interaction.<n>We apply two-stage reinforcement learning (RL) with hybrid rewards, combining verifiable interaction checks and judge-based feedback to improve response continuity.<n> RL further raises source-clean interruption response from 72.5% to 82.5% and reduces synthetic pause barge-in from 26.5% to 9%.
  arXiv  Detail & Related papers  (2026-09-11T09:22:46Z)
• DuplexJail: Safety Alignment Breaks Under Spoken Interruption in Full-Duplex Models [16.95513039797219]
  Fixed-delay interruption raises whole-response attack success rates on AdvBench to 40.3% for PersonaPlex and 48.7% for PersonaPlexRL.<n>The refusal policy reaches 35.6% and 48.6%, respectively, with all trials scored regardless of whether an interruption occurs.<n>These findings identify spoken interruption as a jailbreak attack evaluating safety throughout ongoing full interaction.
  arXiv  Detail & Related papers  (2026-09-08T20:19:24Z)
• ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion [5.90805749495021]
  Full speech models require training yet signals are entangled across speakers in noisy real-world recordings.<n>We present pipeline tracks with stable speaker behavior and evaluate data properties only.<n> downstream gains in full-3.2% model training remain for future work.
  arXiv  Detail & Related papers  (2026-09-08T02:28:43Z)
• DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents [67.07361089429757]
  We measure real-time floor adherence using six real-time speech systems.<n>We find architecture-dependent trade-offs.<n>GPT-Realtime, Mini-o Duplex, Fun-Audio-Chat strongly adhere to persona-consistent content.
  arXiv  Detail & Related papers  (2026-09-03T06:28:02Z)
• TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue [79.83228182492354]
  Speakers in natural conversation take turns speaking and listening, deciding in real time when to take, hold, or yield the floor.<n>We present TurnBench, a benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection.<n>We find end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in backchannel-dense interaction styles.
  arXiv  Detail & Related papers  (2026-08-25T23:14:36Z)
• Hear2Act: Benchmarking When Prosody Should Change What an Assistant Does [27.909684871924004]
  We introduce Hear2Act, a unified evaluation protocol for text and spoken assistants.<n>We evaluate decisions under transcript, audio, and concern-state access.<n>We show that prosody matters when lexical evidence is insufficient.
  arXiv  Detail & Related papers  (2026-08-20T00:16:36Z)
• Chronological Thinking in Full-Duplex Spoken Dialogue Language Models [66.84843878538207]
  Chronological Thinking aims to improve response quality in full SDLMs.<n>No additional latency: once the user stops speaking, the agent halts thinking and begins speaking without further delay.<n>Results: Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations.
  arXiv  Detail & Related papers  (2025-10-02T10:28:11Z)
• X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents [56.64615470513102]
  The Turing test examines whether AIs exhibit human-like behaviour in natural language conversations.<n>Traditional setting limits each participant to one message at a time and requires constant human participation.<n>This paper proposes textbftextscX-Turing, which enhances the original test with a textitburst dialogue pattern.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• Prediction of User Request and Complaint in Spoken Customer-Agent Conversations [0.0]
  This corpus includes a transcription of the spoken conversations and was divided into two sets: Train and Devel sets. We have investigated 14 feature sets: 6 linguistic feature sets, 6 audio feature sets and 2 vocal interaction feature sets. The results show that the linguistic features always give the best results (91.2% for the Request task and 70.3% for the Complaint task)
  arXiv  Detail & Related papers  (2022-07-27T21:29:38Z)
• Responsive Listening Head Generation: A Benchmark Dataset and Baseline [58.168958284290156]
  We define the responsive listening head generation task as the synthesis of a non-verbal head with motions and expressions reacting to the multiple inputs. Unlike speech-driven gesture or talking head generation, we introduce more modals in this task, hoping to benefit several research fields.
  arXiv  Detail & Related papers  (2021-12-27T07:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.