DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization

• URL: http://arxiv.org/abs/2603.09180v1
• Date: Tue, 10 Mar 2026 04:35:22 GMT
• Title: DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization
• Abstract summary: Our key idea is to convert conventional utterance-wise long turns into chunk-turn interactions.<n>To reliably coordinate turn-taking response timing, we introduce a set conversational special control tokens.
• Score: 17.80344425747229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spoken dialog systems with cascaded ASR-LLM-TTS modules retain strong LLM intelligence, but VAD segmentation often forces half-duplex turns and brittle control. On the other hand, VAD-free end-to-end model support full-duplex interaction but is hard to maintain conversational intelligence. In this paper, we present DuplexCascade, a VAD-free cascaded streaming pipeline for full-duplex speech-to-speech dialogue. Our key idea is to convert conventional utterance-wise long turns into chunk-wise micro-turn interactions, enabling rapid bidirectional exchange while preserving the strengths of a capable text LLM. To reliably coordinate turn-taking and response timing, we introduce a set of conversational special control tokens that steer the LLM's behavior under streaming constraints. On Full-DuplexBench and VoiceBench, DuplexCascade delivers state-of-the-art full-duplex turn-taking and strong conversational intelligence among open-source speech-to-speech dialogue systems.

Related papers

• Noise Adaptive Streaming Audio-Visual Speech Token Enhancement for Robust Full-Duplex Spoken Dialogue Models [49.372283081631366]
  AVSTE restores semantic speech tokens from noisy audio and lip video before they reach speech perception.<n>When integrated with Moshi, AVSTE improves average GPT-4ojudged response coherence from 1.42 1.91 under same-data-set speaker interference.
  arXiv  Detail & Related papers  (2026-09-08T07:59:13Z)
• VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents [31.302856029488876]
  VoiceChat-TTS is a low-latency, continuous, and streamable text-to-speech model for interactive agents.<n>The model enables always-on, responsive speech generation while preserving modularity and high speech quality.
  arXiv  Detail & Related papers  (2026-08-13T23:37:07Z)
• Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework [51.969792455913655]
  2 half dialogue systems system system waits for user turn completion before responds.<n>This creates key challenge: anticipate turn timing without sacrificing response quality.<n>We present WildTurnTurn, a largescale, real-world English dataset containing approximately981 hours of multi-turn conversations from face-to-face and telephone conversations, with five turn-level interaction styles, trained on WildTurn LPS-TC rich spoken dynamics that exhibit rich spoken dynamics that are not captured by existing static benchmarks, including we introduce a two-tier evaluation scheme that assesses both in-level timing precision and turn-level interaction quality under realistic streaming constraints,
  arXiv  Detail & Related papers  (2026-08-04T08:22:50Z)
• BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM [37.246890931249496]
  BayLing-Duplex is a native full-generation SpeechLM that decides where to listen, when to speak, and sacrifice when to stop with no auxiliary turn-taking module.<n>It reaches 92% turn-taking success and 100% interruption success on InstructS2S-Eval, while improving speech-response score from 2.17 to 3.39 over Moshi.
  arXiv  Detail & Related papers  (2026-06-12T15:01:51Z)
• Covo-Audio Technical Report [61.09708870154148]
  Covo-Audio, a 7B-end LALM, directly processes continuous audio inputs and generates audio outputs within a single unified architecture.<n>Covo-Audio-Chat, a dialogue-oriented variant, demonstrates semantic strong spoken conversational abilities.
  arXiv  Detail & Related papers  (2026-02-10T14:31:11Z)
• Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems [82.70507055599093]
  We propose a Streaming Chain-of-Thought (CoT) framework for Duplex SDS.<n>We create intermediate targets-aligned user transcripts and system responses for each block.<n>Experiments show that our approach produces more coherent and interpretable responses than existing duplex methods.
  arXiv  Detail & Related papers  (2025-10-02T14:33:05Z)
• SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation [17.56310064245171]
  SALMON-N-omni is the first single standalone full-byte speech LLM that operates without its token transition backbone.<n>It features a novel dynamic thinking mechanism within the LLM backbone, enabling the model to learn when between speaking and listening.<n> SALMON-N-omni demonstrates strong performance in complex conversational scenarios, including turn-taking, backchanneling echo cancellation and context-dependent bargein.
  arXiv  Detail & Related papers  (2025-05-17T08:13:59Z)
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities [93.09944267871163]
  FullDuplexBench is a benchmark that systematically evaluates key interactive behaviors.<n>By releasing our benchmark code we aim to advance spoken dialogue modeling and the development of more natural and engaging SDMs.
  arXiv  Detail & Related papers  (2025-03-06T18:59:16Z)
• LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems [39.144526590642265]
  Speech activity detection (VAD) module efficiently manage dialogue manager (DM) turn-taking in full SDS.<n>By processing speech in short intervals, the VAD enables real-time decision-making, while the core dialogue engine (CDE) is only activated for response generation.
  arXiv  Detail & Related papers  (2025-02-19T23:15:13Z)
• OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation [53.7173034249361]
  End-to-end GPT-based model OmniFlatten capable of effectively modeling complex behaviors inherent natural conversations with low latency.<n>Our approach offers a simple modeling technique and a promising research direction for developing efficient and natural end-to-end full- spoken dialogue systems.
  arXiv  Detail & Related papers  (2024-10-23T11:58:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.