M3-DuplexBench: A Multi-Turn, Multilingual, Multidomain Benchmark for Full-Duplex Spoken Dialogue Models

• URL: http://arxiv.org/abs/2607.29125v1
• Date: Fri, 31 Jul 2026 07:56:25 GMT
• Title: M3-DuplexBench: A Multi-Turn, Multilingual, Multidomain Benchmark for Full-Duplex Spoken Dialogue Models
• Abstract summary: Full spoken dialogue systems (FDSDSs) can listen while speaking, but fair comparisons in multi-turn conversations remain a challenge.<n>FDSDSs M3-DuplexBench supports English and Japanese and covers both casual conversation and multi-turn question answering.
• Score: 41.55514321725997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Full-duplex spoken dialogue systems (FDSDSs) can listen while speaking, enabling natural behaviors such as smooth turn-taking, backchannel handling, and user barge-in handling. However, fair comparisons in multi-turn conversations remain a challenge. In addition, existing benchmarks provide limited coverage of languages and dialogue domains. We propose M3-DuplexBench, a multi-turn, multilingual, multidomain benchmark for FDSDSs. M3-DuplexBench supports English and Japanese and covers both casual conversation and multi-turn question answering. In addition, we evaluate models under multiple dialogue context settings, including single-turn, user-only, and teacher-forced full-context settings, to analyze how dialogue history affects model behavior. Experiments with recent FDSDSs reveal model-specific turn-taking characteristics, clear performance gaps across languages and domains, and mixed effects of dialogue context.

Related papers

• S2Dialog: Multimodal Dialogue Retrieval with Semantic and Acoustic-Style Modeling [10.413079982380555]
  We propose S2Dialog, a unified framework for dialogue-level semantic-style retrieval from multimodal dialogue banks.<n>S2Dialog consists of a Dialogue-level Textual Retriever and a Dialogue-level Acoustic Retriever, which encode the textual and acoustic modalities of a dialogue into dialogue-level representations.<n>Experiments on the multimodal dialogue dataset DailyTalk demonstrate that S2Dialog achieves outstanding retrieval performance.
  arXiv  Detail & Related papers  (2026-08-14T07:19:24Z)
• MOSS-TTSD: Text to Spoken Dialogue Generation [62.04179716555789]
  We present MOSS-TTSD, a spoken dialogue synthesis model for expressive, multi-party conversational speech across multiple languages.<n>With enhanced long-context modeling, MOSS-TTSD generates long-form spoken conversations from dialogue scripts with explicit speaker tags.<n>The model supports various mainstream languages, including English and Chinese, and is adapted to several long-form scenarios.
  arXiv  Detail & Related papers  (2026-03-20T08:23:31Z)
• ContextQFormer: A New Context Modeling Method for Multi-Turn Multi-Modal Conversations [38.40471808648207]
  We introduce a context modeling module, termed ContextQFormer, to enhance the presentation of contextual information.<n>To facilitate further research, we build a new multi-turn multi-modal dialogue dataset (TMDialog) for pre-training, instruction-tuning, and evaluation.<n>In addition, ContextQFormer is compared with three baselines on TMDialog and experimental results illustrate that ContextQFormer achieves an improvement of 2%-4% in available rate over baselines.
  arXiv  Detail & Related papers  (2025-05-29T05:41:26Z)
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities [93.09944267871163]
  FullDuplexBench is a benchmark that systematically evaluates key interactive behaviors.<n>By releasing our benchmark code we aim to advance spoken dialogue modeling and the development of more natural and engaging SDMs.
  arXiv  Detail & Related papers  (2025-03-06T18:59:16Z)
• Can MLLMs Generalize to Multi-Party dialog? Exploring Multilingual Response Generation in Complex Scenarios [8.131774353504472]
  We introduce XMP, a high-quality parallel Multilingual dataset sourced from Multi-party Podcast dialogues.<n>Most samples in the dataset feature three or more participants, discussing a wide range of topics.<n>We find that, R1: MLLMs fail to generalize to multi-party setting, R2 Fine-tuning on XMP improves only marginally, with the 70B model achieving at most a 1% absolute gain over its 8B counterpart.
  arXiv  Detail & Related papers  (2025-01-20T04:33:03Z)
• OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation [53.7173034249361]
  End-to-end GPT-based model OmniFlatten capable of effectively modeling complex behaviors inherent natural conversations with low latency.<n>Our approach offers a simple modeling technique and a promising research direction for developing efficient and natural end-to-end full- spoken dialogue systems.
  arXiv  Detail & Related papers  (2024-10-23T11:58:58Z)
• Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation [55.043492250775294]
  We introduce a novel Face-to-Face spoken dialogue model. It processes audio-visual speech from user input and generates audio-visual speech as the response. We also introduce MultiDialog, the first large-scale multimodal spoken dialogue corpus.
  arXiv  Detail & Related papers  (2024-06-12T04:48:36Z)
• SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation [123.37377363355363]
  SPACE-3 is a novel unified semi-supervised pre-trained conversation model learning from large-scale dialog corpora. It can be effectively fine-tuned on a wide range of downstream dialog tasks. Results show that SPACE-3 achieves state-of-the-art performance on eight downstream dialog benchmarks.
  arXiv  Detail & Related papers  (2022-09-14T14:17:57Z)
• Fusing task-oriented and open-domain dialogues in conversational agents [12.338220374261343]
  Two dialogue modes can potentially be intertwined together seamlessly in the same conversation, as easily done by a friendly human assistant. Our paper addresses this problem of fusing TODs and ODDs in multi-turn dialogues. It features inter-mode contextual dependency, i.e., the dialogue turns from the two modes depend on each other.
  arXiv  Detail & Related papers  (2021-09-09T09:48:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.