StyleBench: Evaluating Speech Language Models on Conversational Speaking Style Control

• URL: http://arxiv.org/abs/2603.07599v1
• Date: Sun, 08 Mar 2026 11:58:29 GMT
• Title: StyleBench: Evaluating Speech Language Models on Conversational Speaking Style Control
• Abstract summary: Speech language models (SLMs) have significantly extended the interactive capability of text-based Large Language Models (LLMs)<n>Our results reveal the performance gaps between leading SLMs and omni language models (OLMs)
• Score: 31.566245098322224
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech language models (SLMs) have significantly extended the interactive capability of text-based Large Language Models (LLMs) by incorporating paralinguistic information. For more realistic interactive experience with customized styles, current SLMs have managed to interpret and control speaking style intensity from user prompts during the dialogue process. However, there remains a lack of systematic benchmarks that quantifies and evaluates the style intensity control ability in conversations. In this paper, we propose StyleBench, a multi-turn dialogue benchmark for comprehensively evaluating the style intensity control ability across four dimensions: emotion, speed, volume, and pitch. Our results reveal the performance gaps between leading SLMs and omni language models (OLMs), suggesting the underlying reasons and promising approaches for future exploration.

Related papers

• Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning [57.22705949022221]
  RoleJudge is an evaluation framework that leverages audio large language models to assess the alignment between speech and character.<n>We introduce RoleChat, the first voice role-playing evaluation dataset enriched with chain-of-thought reasoning annotations.
  arXiv  Detail & Related papers  (2026-04-15T12:39:03Z)
• VStyle: A Benchmark for Voice Style Adaptation with Spoken Instructions [66.93932684284695]
  Spoken language models (SLMs) have emerged as a unified paradigm for speech understanding and generation.<n>We introduce Voice Style Adaptation (VSA), a new task that examines whether SLMs can modify their speaking style.<n>We present VStyle, a benchmark covering four categories of speech generation: acoustic attributes, natural language instruction, role play, and implicit empathy.<n>We also introduce the Large Audio Language Model as a Judge (LALM as a Judge) framework, which progressively evaluates outputs along textual faithfulness, style adherence, and naturalness.
  arXiv  Detail & Related papers  (2025-09-09T14:28:58Z)
• TELEVAL: A Dynamic Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarios [47.08170350061827]
  Spoken language models (SLMs) have seen rapid progress in recent years, along with the development of numerous benchmarks for evaluating their performance.<n>Most existing benchmarks primarily focus on evaluating whether SLMs can perform complex tasks comparable to those tackled by large language models (LLMs)<n>We propose a benchmark specifically designed to evaluate SLMs' effectiveness as conversational agents in realistic Chinese interactive settings.
  arXiv  Detail & Related papers  (2025-07-24T03:23:55Z)
• MultiVox: A Benchmark for Evaluating Voice Assistants for Multimodal Interactions [70.93364531054273]
  We introduce MultiVox, the first benchmark to evaluate the ability of voice assistants to integrate spoken and visual cues.<n>Specifically, MultiVox includes 1000 human-annotated and recorded speech dialogues that encompass diverse paralinguistic features.<n>Our evaluation on 10 state-of-the-art models reveals that, although humans excel at these tasks, current models consistently struggle to produce contextually grounded responses.
  arXiv  Detail & Related papers  (2025-07-14T23:20:42Z)
• Aligning Spoken Dialogue Models from User Interactions [55.192134724622235]
  We propose a novel preference alignment framework to improve spoken dialogue models on realtime conversations from user interactions.<n>We create a dataset of more than 150,000 preference pairs from raw multi-turn speech conversations annotated with AI feedback.<n>Our findings shed light on the importance of a well-calibrated balance among various dynamics, crucial for natural real-time speech dialogue systems.
  arXiv  Detail & Related papers  (2025-06-26T16:45:20Z)
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities [93.09944267871163]
  FullDuplexBench is a benchmark that systematically evaluates key interactive behaviors.<n>By releasing our benchmark code we aim to advance spoken dialogue modeling and the development of more natural and engaging SDMs.
  arXiv  Detail & Related papers  (2025-03-06T18:59:16Z)
• SAGE: Steering Dialog Generation with Future-Aware State-Action Augmentation [9.95917154889491]
  We present a novel approach called SAGE that uses latent variables to control long-horizon behavior in dialogue generation.<n>At the core of our method is the State-Action Chain (SAC), which augments standard language model fine-tuning.<n>Our experimental results show that models trained with this approach demonstrate improved performance in emotional intelligence metrics.
  arXiv  Detail & Related papers  (2025-03-04T22:45:24Z)
• Generative Expressive Conversational Speech Synthesis [47.53014375797254]
  Conversational Speech Synthesis (CSS) aims to express a target utterance with the proper speaking style in a user-agent conversation setting. We propose a novel generative expressive CSS system, termed GPT-Talker. We transform the multimodal information of the multi-turn dialogue history into discrete token sequences and seamlessly integrate them to form a comprehensive user-agent dialogue context.
  arXiv  Detail & Related papers  (2024-07-31T10:02:21Z)
• StyleChat: Learning Recitation-Augmented Memory in LLMs for Stylized Dialogue Generation [43.29667566560533]
  We introduce a stylized dialogue dataset StyleEval with 38 styles by leveraging the generative power of Large Language Models (LLMs) We propose the stylized dialogue framework StyleChat via recitation-augmented memory strategy and multi-task style learning strategy to promote generalization ability.
  arXiv  Detail & Related papers  (2024-03-18T03:26:18Z)
• Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations [65.29513437838457]
  Even if two current turns are the same sentence, their responses might still differ when they are spoken in different styles. We propose Spoken-LLM framework that can model the linguistic content and the speaking styles. We train Spoken-LLM using the StyleTalk dataset and devise a two-stage training pipeline to help the Spoken-LLM better learn the speaking styles.
  arXiv  Detail & Related papers  (2024-02-20T07:51:43Z)
• Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation [46.93969003104427]
  This paper introduces an extensive speech-text LLM framework, the Unified Spoken Dialog Model (USDM)<n>USDM is designed to generate coherent spoken responses with naturally occurring prosodic features relevant to the given input speech.<n>Our approach effectively generates natural-sounding spoken responses, surpassing previous and cascaded baselines.
  arXiv  Detail & Related papers  (2024-02-08T14:35:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.