Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework

• URL: http://arxiv.org/abs/2608.28630v1
• Date: Tue, 04 Aug 2026 08:22:50 GMT
• Title: Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework
• Abstract summary: 2 half dialogue systems system system waits for user turn completion before responds.<n>This creates key challenge: anticipate turn timing without sacrificing response quality.<n>We present WildTurnTurn, a largescale, real-world English dataset containing approximately981 hours of multi-turn conversations from face-to-face and telephone conversations, with five turn-level interaction styles, trained on WildTurn LPS-TC rich spoken dynamics that exhibit rich spoken dynamics that are not captured by existing static benchmarks, including we introduce a two-tier evaluation scheme that assesses both in-level timing precision and turn-level interaction quality under realistic streaming constraints,
• Score: 51.969792455913655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compared with half-duplex dialogue systems where the system waits for user turn completion before it responds, natural full-duplex dialogue systems require agents to act proactively in real time, including timely interruptions and backchannels. This creates a key challenge: improving turn timing without sacrificing response quality. To address limitations in realistic proactive turn-taking, we build a generalized style-aware full-duplex framework with three key components. Firstly, we propose LPS-TC, a Lightweight Proactive Speech Turn Controller for plug-and-play integration. It features a fine-grained action space covering both reactive and proactive turn behaviors, enabling half-duplex models with full-duplex capabilities and enhancing existing full-duplex models with superior timing control. Secondly, we construct WildTurn, a large-scale, real-world English dataset containing approximately 2,981 hours of filtered multi-turn stereo conversations from face-to-face and telephone conversations, annotated with five turn-taking and five backchanneling styles. Trained on WildTurn, LPS-TC exhibits rich spoken dynamics that are not captured by existing static full-duplex benchmarks. Thirdly, we introduce a two-tier evaluation scheme that assesses both chunk-level timing precision and turn-level interaction quality under realistic streaming constraints. Our experiments, integrating LPS-TC with half-duplex models like Qwen2.5-Omni and full-duplex models like Freeze-Omni, showcase its superior performance in timing appropriateness and response quality. Our framework also demonstrates fine-grained style controllability and strong generalizability, enabling more natural and human-like spoken interactions.

Related papers

• Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models [54.83270723528963]
  We propose a method to add streaming capabilities to an existing S2S model.<n>Our approach requires minimal additional parameters and no significant changes to the base S2S model.
  arXiv  Detail & Related papers  (2026-09-14T15:44:18Z)
• BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM [37.246890931249496]
  BayLing-Duplex is a native full-generation SpeechLM that decides where to listen, when to speak, and sacrifice when to stop with no auxiliary turn-taking module.<n>It reaches 92% turn-taking success and 100% interruption success on InstructS2S-Eval, while improving speech-response score from 2.17 to 3.39 over Moshi.
  arXiv  Detail & Related papers  (2026-06-12T15:01:51Z)
• Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models [53.470209949659115]
  We propose a post-training alignment method that improves full spoken dialogue models through RL.<n>We address the four canonical axes of interactivity: handling turn-taking, back-channel pause, and user interruption.<n>An extra-based reward for response quality prevents semantic degradation.
  arXiv  Detail & Related papers  (2026-06-09T17:46:55Z)
• MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction [76.4461698685681]
  Mini-o 4.5 is our latest effort towards human-level realtime streaming interaction.<n>It is based on Omni-CPM, a unified streaming framework that aligns omni-modal perception and outputs along a shared temporal axis.<n>With a total of 9B parameters, Mini-o 4.5 approaches Gemini 2.5 Flash in vision computation capabilities, delivering stateofthe-art open computation performance scale.
  arXiv  Detail & Related papers  (2026-04-30T04:05:43Z)
• DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization [17.80344425747229]
  Our key idea is to convert conventional utterance-wise long turns into chunk-turn interactions.<n>To reliably coordinate turn-taking response timing, we introduce a set conversational special control tokens.
  arXiv  Detail & Related papers  (2026-03-10T04:35:22Z)
• Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems [82.70507055599093]
  We propose a Streaming Chain-of-Thought (CoT) framework for Duplex SDS.<n>We create intermediate targets-aligned user transcripts and system responses for each block.<n>Experiments show that our approach produces more coherent and interpretable responses than existing duplex methods.
  arXiv  Detail & Related papers  (2025-10-02T14:33:05Z)
• RoboEgo System Card: An Omnimodal Model with Native Full Duplexity [48.52383812141669]
  RoboEgo (alias: FLM-Ego) is a unified model system designed to address both challenges.<n>FLM-Ego incorporates backbone and algorithms that support fullity, achieving a theoretical duplex of 80 ms latency.
  arXiv  Detail & Related papers  (2025-06-02T17:53:10Z)
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities [93.09944267871163]
  FullDuplexBench is a benchmark that systematically evaluates key interactive behaviors.<n>By releasing our benchmark code we aim to advance spoken dialogue modeling and the development of more natural and engaging SDMs.
  arXiv  Detail & Related papers  (2025-03-06T18:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.