D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones
Abstract Overview
This paper presents Dialogue-to-Duplex (D2), a framework for converting pretrained half-duplex spoken language models into full-duplex models that can listen and speak simultaneously at fine temporal granularity. The method has two main stages: decoder-free streaming encoder distillation to adapt the original audio encoder for low-latency chunked inference, and supervised fine-tuning on temporally aligned duplex dialogue sequences to learn interruption handling and turn control. The authors apply D2 to Qwen3-Omni and LLaMA-Omni 2, and systematically study how chunk size affects responsiveness, retained general capability, and computation cost. The reported evidence shows that D2 can preserve much of the backbone models’ spoken-language capability while enabling strong turn-taking and interruption behavior in real time.
Novelty
The distinctive contribution is a practical HD-to-FD conversion framework that does not require the original training decoder or transcription labels for streaming adaptation, using representation distillation instead. The work also emphasizes fine-grained duplex operation at small chunk sizes and provides a systematic analysis of chunk-size trade-offs across interaction quality, capability retention, and inference efficiency.
Results
On Qwen3-Omni at a 12.5 Hz update rate (80 ms chunks), D2 achieves 99.2% turn-taking success, 99.5% interruption success, 521 ms turn-taking latency, 403 ms interruption latency, and a 70.1 VoiceBench score. Across tested chunk sizes, Qwen3-Omni-D2 attains the best reported VoiceBench results among the evaluated full-duplex models, while computation measurements indicate real-time inference on a single RTX PRO 6000 GPU. Ablations further show that Stage-I encoder adaptation improves response quality and capability retention, and that control-signal weighting materially supports turn control.
Key Points
- D2 converts strong half-duplex spoken language backbones into full-duplex systems through streaming encoder distillation followed by duplex supervised fine-tuning.
- The method is demonstrated on both Qwen3-Omni and LLaMA-Omni 2, with chunk size serving as a key control knob for latency, pause handling, and retained capability.
- Qwen3-Omni-D2 shows near-perfect turn-taking and interruption rates and state-of-the-art VoiceBench performance among the evaluated full-duplex baselines, while still running in real time.