Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

• URL: http://arxiv.org/abs/2604.04847v1
• Date: Mon, 06 Apr 2026 16:46:52 GMT
• Title: Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency
• Abstract summary: We introduce Full-Duplex-Bench-v3 (FDB-v3), a benchmark for evaluating spoken language models under naturalistic speech conditions and multi-step tool use.<n>Unlike prior work, our dataset consists entirely of real human audio annotated for five disfluency categories, paired with scenarios requiring chained API calls across four task domains.<n>We evaluate six model configurations -- GPT-Realtime, Gemini Live 2.5, Gemini Live 3.1, Grok, Ultravox v0.7, and a traditional Cascaded pipeline (Whisper$rightarrow$rightarrow$TTS) -- across accuracy, latency, and turn-
• Score: 61.68376148916503
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We introduce Full-Duplex-Bench-v3 (FDB-v3), a benchmark for evaluating spoken language models under naturalistic speech conditions and multi-step tool use. Unlike prior work, our dataset consists entirely of real human audio annotated for five disfluency categories, paired with scenarios requiring chained API calls across four task domains. We evaluate six model configurations -- GPT-Realtime, Gemini Live 2.5, Gemini Live 3.1, Grok, Ultravox v0.7, and a traditional Cascaded pipeline (Whisper$\rightarrow$GPT-4o$\rightarrow$TTS) -- across accuracy, latency, and turn-taking dimensions. GPT-Realtime leads on Pass@1 (0.600) and interruption avoidance (13.5\%); Gemini Live 3.1 achieves the fastest latency (4.25~s) but the lowest turn-take rate (78.0\%); and the Cascaded baseline, despite a perfect turn-take rate, incurs the highest latency (10.12~s). Across all systems, self-correction handling and multi-step reasoning under hard scenarios remain the most consistent failure modes.

Related papers

• WaVeFuse: Regime-Adaptive Equity Index Forecasting via Channel-Wise Wavelet Denoising and Vertical Attention Fusion [0.0]
  WaVeFuse is a hybrid Deep Learning for equity forecasting framework.<n>It addresses propagation of technical indicators (TIs), channel-indiscriminate decomposition, and static multi-branch fusion.<n>WaVeFuse delivers a computationally efficient, regime-robust framework suitable for research and decision-support deployment.
  arXiv  Detail & Related papers  (2026-09-13T18:50:33Z)
• Realtime-Venus: A full-duplex interaction system with asynchronous delegation [59.69745588222872]
  We present Realtime, Realtime Omni audio visual interaction and Realtime-Audio Spoken for spoken interaction.<n>Each model serves as complete conversational, integrating continuous perception, conversational control, and native speech generation.
  arXiv  Detail & Related papers  (2026-09-12T08:56:50Z)
• Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework [51.969792455913655]
  2 half dialogue systems system system waits for user turn completion before responds.<n>This creates key challenge: anticipate turn timing without sacrificing response quality.<n>We present WildTurnTurn, a largescale, real-world English dataset containing approximately981 hours of multi-turn conversations from face-to-face and telephone conversations, with five turn-level interaction styles, trained on WildTurn LPS-TC rich spoken dynamics that exhibit rich spoken dynamics that are not captured by existing static benchmarks, including we introduce a two-tier evaluation scheme that assesses both in-level timing precision and turn-level interaction quality under realistic streaming constraints,
  arXiv  Detail & Related papers  (2026-08-04T08:22:50Z)
• VisualClaw: A Real-Time, Personalized Agent for the Physical World [76.89211120690028]
  We present VisualClaw, a self-evolving multimodal agent built around two principles.<n>First, hybrid encoding reduces deployment cost by filtering less informative streaming frames with a cascaded gate.<n>Second, skill evolution lets the agent learn from failures, producing skill-bank updates that help future questions.
  arXiv  Detail & Related papers  (2026-06-15T06:58:22Z)
• Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference [5.685908474026849]
  High-quality automatic speech recognition (ASR) on edge devices requires models that jointly optimize accuracy, latency, and memory footprint while operating entirely on CPU without GPU acceleration.<n>We conduct a systematic empirical study of state-of-the-art ASR architectures, encompassing encoder-decoder, transducer, and LLM-based paradigms, evaluated across batch, chunked, and streaming inference modes.<n>Our recommended configuration, the int4 k-quant variant, achieves 8.20% average streaming WER across eight standard benchmarks, running comfortably faster than real-time on CPU with 0.56 s algorithmic latency.
  arXiv  Detail & Related papers  (2026-04-16T00:04:32Z)
• How Small Can 6G Reason? Scaling Tiny Language Models for AI-Native Networks [3.099103925863002]
  We study the scaling behavior and deployment efficiency of compact language models for network-level semantic reasoning in AI-native 6G systems.<n>We evaluate models ranging from 135M (SmolLM2-135M) to 7B parameters (Qwen2.5-7B), including mid-scale architectures such as Llama-3.2-1B, Granite-1B, and Qwen2.5-3B.
  arXiv  Detail & Related papers  (2026-03-02T18:19:49Z)
• Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters [169.7981969517903]
  Step 3.5 Flash bridges frontier-level agentic intelligence and computational efficiency.<n>We focus on what matters most when building agents: sharp reasoning and fast, reliable execution.
  arXiv  Detail & Related papers  (2026-02-11T07:53:51Z)
• Qwen3-ASR Technical Report [71.87071808763484]
  We introduce Qwen3-ASR family, which includes two powerful all-in-one speech recognition models and a novel non-autoregressive speech forced alignment model.<n>Qwen3-ASR-1.7B and Qwen3-ASR-0.6B are ASR models that support language identification and ASR for 52 languages and dialects.
  arXiv  Detail & Related papers  (2026-01-29T06:58:13Z)
• Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression [0.0]
  Two adaptive compression techniques are proposed to integrate algorithmic innovations with hardware-aware optimizations.<n>On event-based vision tasks, STTF reduces average token count by 84%.<n>ANC cuts FLOPs by up to 90% in low-motion scenes.
  arXiv  Detail & Related papers  (2025-11-23T15:43:00Z)
• Efficient Hate Speech Detection: A Three-Layer LoRA-Tuned BERTweet Framework [0.0]
  This paper addresses the challenge of developing computationally efficient hate speech detection systems.<n>We propose a novel three-layer framework that combines rule-based pre-filtering with a parameter-efficient LoRA-tuned BERTweet model.<n>Our approach achieves 94% of the performance of state-of-the-art large language models like SafePhi.
  arXiv  Detail & Related papers  (2025-11-08T15:47:18Z)
• AMAuT: A Flexible and Efficient Multiview Audio Transformer Framework Trained from Scratch [0.3728263002609659]
  This paper introduces the Augmentation-driven Multiview Audio Transformer (AMAuT)<n>AMAuT eliminates the dependency on pre-trained weights while supporting arbitrary sample rates and audio lengths.<n> Experiments on five public benchmarks, AudioMNIST, SpeechCommands V1 & V2, VocalSound, and CochlScene, show that AMAuT achieves accuracies up to 99.8%.
  arXiv  Detail & Related papers  (2025-10-22T08:41:59Z)
• HELENA: High-Efficiency Learning-based channel Estimation using dual Neural Attention [0.0]
  HELENA is a compact deep learning model that combines a lightweight convolutional backbone with two efficient attention mechanisms.<n>HELENA reduces inference time by 45.0% (0.175,ms vs. 0.318,ms)
  arXiv  Detail & Related papers  (2025-06-16T12:21:27Z)
• Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision [83.0622534215881]
  This work proposes an industry-level omni-modal large language model (LLM) pipeline that integrates auditory, visual, and linguistic modalities.<n>Our pipeline consists of three main components: First, a modular framework enabling flexible configuration of various encoder-LLM-decoder architectures.<n>Second, a lightweight training strategy that pre-trains audio-language alignment on the state-of-the-art vision-language model Qwen2.5-VL.<n>Third, an audio synthesis pipeline that generates high-quality audio-text data from diverse real-world scenarios.
  arXiv  Detail & Related papers  (2025-02-26T17:26:36Z)
• Baichuan-Omni-1.5 Technical Report [78.49101296394218]
  Baichuan- Omni-1.5 is an omni-modal model that not only has omni-modal understanding capabilities but also provides end-to-end audio generation capabilities.<n>We establish a comprehensive data cleaning and synthesis pipeline for multimodal data, obtaining about 500B high-quality data.<n>Second, an audio-tokenizer has been designed to capture both semantic and acoustic information from audio, enabling seamless integration and enhanced compatibility with MLLM.
  arXiv  Detail & Related papers  (2025-01-26T02:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.