ECHO: A Matched-Contrast Benchmark for Context-Sensitive Turn-Taking in Full-Duplex Dialogue

• URL: http://arxiv.org/abs/2609.17360v1
• Date: Tue, 15 Sep 2026 15:58:48 GMT
• Title: ECHO: A Matched-Contrast Benchmark for Context-Sensitive Turn-Taking in Full-Duplex Dialogue
• Abstract summary: We develop diagnostic benchmark for Chinese full spoken dialogue turn-taking pairs pairs examples.<n>We introduce pair accuracy, which requires correct decisions for both members of a pair and assigns no credit to constant-action policies.<n>These findings demonstrate that interruption-only evaluation can distinguish practical turntaking reliability.
• Score: 19.857222937196745
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Full-duplex spoken dialogue systems must distinguish interruptions that require yielding the floor from backchannels that permit continued speaking. Existing benchmarks typically evaluate events independently and may therefore reward fixed action preferences rather than context-sensitive decisions. We introduce ECHO, a paired diagnostic benchmark for Chinese full-duplex turn-taking. ECHO pairs examples with the same overlap transcript but contrasting preceding multi-turn dialogue contexts, with one requiring Yield and the other Keep. It additionally includes off-talk examples for diagnosing unnecessary yielding. We introduce pair accuracy, which requires correct decisions on both members of a pair and assigns no credit to constant-action policies. Experiments on multiple full-duplex systems show that most exhibit a pronounced bias toward \textsc{Yield}, performing substantially better on interruptions than on backchannels, while another system remains comparatively balanced. These findings demonstrate that interruption-only evaluation can overestimate practical turn-taking reliability. ECHO and its metadata will be publicly released.

Related papers

• ANCHOR: Autoregressive Non-intrusive Chunk-Ordered Refinement for Joint Multi-Resolution Speech Quality Modeling [62.983372215479925]
  Streaming and generative systems require incremental estimation from partial audio.<n>We propose ANCHOR, reformulating incremental assessment as a multi-resolution autore task.<n> Experiments show substantial robustness under partial input, including a 48% PLCMOS error reduction on 2-second prefixes.
  arXiv  Detail & Related papers  (2026-06-08T22:46:30Z)
• Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction [4.273094752480624]
  We introduce prover-verifier deliberation (PVD), an inference-time protocol grounded in interactive proof theory.<n>PVD produces both an answer and a structured confidence verdict, allowing a system to report high-confidence answers while abstaining on uncertain cases.
  arXiv  Detail & Related papers  (2026-05-24T15:23:27Z)
• Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought [49.53567098922619]
  We propose Separate First, Fuse Later (SFFL) as an audio-visual reasoning framework to reduce cross-modal interference.<n>SFFL enforces modality-specific chain-of-thought reasoning, producing separate audio and visual reasoning traces and integrating evidence for answering.<n> Experiments demonstrate consistent improvements in both accuracy and robustness, yielding an average relative gain of 5.16% on general AVQA benchmarks and 11.17% on a cross-modal hallucination benchmark.
  arXiv  Detail & Related papers  (2026-05-11T02:50:27Z)
• To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition [87.22988227382329]
  Multimodal emotion recognition (MER) benefits from combining text, audio, and vision, yet standard fusion often fails when modalities conflict.<n>We propose Dual-Path Conflict Resolution (DCR), a unified framework that learns when to fuse and when to drop modalities.
  arXiv  Detail & Related papers  (2026-05-06T13:11:33Z)
• BiCon-Gate: Consistency-Gated De-colloquialisation for Dialogue Fact-Checking [9.866690791407464]
  We propose a conservative rewrite candidate for each response claim via staged de-colloquialisation.<n>We then introduce BiCon-Gate, a semantics-aware consistency gate that selects the rewrite candidate only when it is semantically supported by the dialogue context.<n>This gated selection stabilises downstream fact-checking and yields gains in both evidence retrieval and fact verification.
  arXiv  Detail & Related papers  (2026-04-15T20:06:24Z)
• Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness [45.06366615980232]
  We introduce SDiaReward, an end-to-end multi-turn reward model trained on SDiaReward-Dataset.<n>It operates directly on full multi-turn speech episodes and is optimized with pairwise preference supervision.<n>Experiments demonstrate that SDiaReward achieves state-of-the-art pairwise preference accuracy.
  arXiv  Detail & Related papers  (2026-03-16T06:39:30Z)
• Measuring Language Model Hallucinations Through Distributional Correctness [7.106986689736826]
  A novel evaluation metric, the Distributional Correctness Score (DCS), is introduced to solve this problem.<n>DCS distinguishes between harmful overconfidence in wrong answers and uncertainty expressed through abstention, providing scores in an interpretable default range.<n>DCS is demonstrated to offer a more nuanced and aligned evaluation paradigm that incentivises models to express genuine uncertainty rather than guessing.
  arXiv  Detail & Related papers  (2025-10-05T17:50:42Z)
• Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems [82.70507055599093]
  We propose a Streaming Chain-of-Thought (CoT) framework for Duplex SDS.<n>We create intermediate targets-aligned user transcripts and system responses for each block.<n>Experiments show that our approach produces more coherent and interpretable responses than existing duplex methods.
  arXiv  Detail & Related papers  (2025-10-02T14:33:05Z)
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities [93.09944267871163]
  FullDuplexBench is a benchmark that systematically evaluates key interactive behaviors.<n>By releasing our benchmark code we aim to advance spoken dialogue modeling and the development of more natural and engaging SDMs.
  arXiv  Detail & Related papers  (2025-03-06T18:59:16Z)
• End-to-end contextual asr based on posterior distribution adaptation for hybrid ctc/attention system [61.148549738631814]
  End-to-end (E2E) speech recognition architectures assemble all components of traditional speech recognition system into a single model. Although it simplifies ASR system, it introduces contextual ASR drawback: the E2E model has worse performance on utterances containing infrequent proper nouns. We propose to add a contextual bias attention (CBA) module to attention based encoder decoder (AED) model to improve its ability of recognizing the contextual phrases.
  arXiv  Detail & Related papers  (2022-02-18T03:26:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.