SpeechJudge: Towards Human-Level Judgment for Speech Naturalness

• URL: http://arxiv.org/abs/2511.07931v1
• Date: Wed, 12 Nov 2025 01:29:01 GMT
• Title: SpeechJudge: Towards Human-Level Judgment for Speech Naturalness
• Abstract summary: SpeechJudge is a suite comprising a dataset, a benchmark, and a reward model.<n>We present SpeechJudge-Data, a large-scale human feedback corpus of 99K speech pairs.<n> SpeechJudge-Eval is a challenging benchmark for speech naturalness judgment.<n>We develop SpeechJudge-GRM, a generative reward model based on Qwen2.5- Omni-7B.
• Score: 25.566620976752443
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Aligning large generative models with human feedback is a critical challenge. In speech synthesis, this is particularly pronounced due to the lack of a large-scale human preference dataset, which hinders the development of models that truly align with human perception. To address this, we introduce SpeechJudge, a comprehensive suite comprising a dataset, a benchmark, and a reward model centered on naturalness--one of the most fundamental subjective metrics for speech synthesis. First, we present SpeechJudge-Data, a large-scale human feedback corpus of 99K speech pairs. The dataset is constructed using a diverse set of advanced zero-shot text-to-speech (TTS) models across diverse speech styles and multiple languages, with human annotations for both intelligibility and naturalness preference. From this, we establish SpeechJudge-Eval, a challenging benchmark for speech naturalness judgment. Our evaluation reveals that existing metrics and AudioLLMs struggle with this task; the leading model, Gemini-2.5-Flash, achieves less than 70% agreement with human judgment, highlighting a significant gap for improvement. To bridge this gap, we develop SpeechJudge-GRM, a generative reward model (GRM) based on Qwen2.5-Omni-7B. It is trained on SpeechJudge-Data via a two-stage post-training process: Supervised Fine-Tuning (SFT) with Chain-of-Thought rationales followed by Reinforcement Learning (RL) with GRPO on challenging cases. On the SpeechJudge-Eval benchmark, the proposed SpeechJudge-GRM demonstrates superior performance, achieving 77.2% accuracy (and 79.4% after inference-time scaling @10) compared to a classic Bradley-Terry reward model (72.7%). Furthermore, SpeechJudge-GRM can be also employed as a reward function during the post-training of speech generation models to facilitate their alignment with human preferences.

Related papers

• Preference learning in shades of gray: Interpretable and bias-aware reward modeling for human preferences [8.680081568962999]
  This study investigates the limits of current approaches to learning human preferences in language models.<n>We propose a feature-augmented framework to better capture the multidimensional nature of human judgment.
  arXiv  Detail & Related papers  (2026-04-01T18:26:16Z)
• Neural networks for Text-to-Speech evaluation [35.277885003866466]
  Speech-to-Speech systems need to deliver human-perceived quality at scale.<n>Human subjective evaluation protocols such as Mean Opinion Score (MOS) and Side-by-Side (SBS) comparisons remain the de facto gold standards.<n>This study addresses these barriers by formulating, and implementing a suite of novel neural models.
  arXiv  Detail & Related papers  (2026-03-17T16:07:15Z)
• GSRM: Generative Speech Reward Model for Speech RLHF [28.19230481779836]
  Generative Speech Reward Model (GSRM) is a reasoning-centric reward model tailored for speech.<n>The GSRM is trained to decompose speech naturalness evaluation into an interpretable acoustic feature extraction stage.<n> Experiments show that GSRM substantially outperforms existing speech naturalness predictors.
  arXiv  Detail & Related papers  (2026-02-14T21:22:55Z)
• Reward Modeling from Natural Language Human Feedback [77.75758630455357]
  Reinforcement Learning with Verifiable reward (RLVR) on preference data has become the mainstream approach for training Generative Reward Models (GRMs)<n>In this paper, we demonstrate that such binary classification tasks make GRMs susceptible to guessing correct outcomes without sound critiques.<n>We propose Reward Modeling from Natural Language Human Feedback (RM-NLHF), which leverages natural language feedback to obtain process reward signals.
  arXiv  Detail & Related papers  (2026-01-12T09:23:43Z)
• Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy [26.455112415445146]
  We introduce Skywork-Reward-V2, a suite of eight reward models ranging from 0.6B to 8B parameters, trained on a subset of 26 million preference pairs from SynPref-40M.<n>We demonstrate that Skywork-Reward-V2 is versatile across a wide range of capabilities, including alignment with human preferences, objective correctness, safety, resistance to stylistic biases, and best-of-N scaling.
  arXiv  Detail & Related papers  (2025-07-02T04:40:29Z)
• Audio Large Language Models Can Be Descriptive Speech Quality Evaluators [46.765203628127345]
  We introduce the first natural language-based speech evaluation corpus, generated from authentic human ratings.<n>This corpus offers detailed analysis across multiple dimensions and identifies causes of quality degradation.<n>We propose an alignment approach with LLM distillation (ALLD) to guide the audio LLM in extracting relevant information from raw speech.
  arXiv  Detail & Related papers  (2025-01-27T22:47:51Z)
• Speechworthy Instruction-tuned Language Models [71.8586707840169]
  We show that both prompting and preference learning increase the speech-suitability of popular instruction-tuned LLMs. We share lexical, syntactical, and qualitative analyses to showcase how each method contributes to improving the speech-suitability of generated responses.
  arXiv  Detail & Related papers  (2024-09-23T02:34:42Z)
• SpeechAlign: Aligning Speech Generation to Human Preferences [51.684183257809075]
  We introduce SpeechAlign, an iterative self-improvement strategy that aligns speech language models to human preferences. We show that SpeechAlign can bridge the distribution gap and facilitate continuous self-improvement of the speech language model.
  arXiv  Detail & Related papers  (2024-04-08T15:21:17Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• The DiffuseStyleGesture+ entry to the GENEA Challenge 2023 [16.297790031478634]
  We introduce the DiffuseStyleGesture+, our solution for the Generation and Evaluation of Non-verbal Behavior for Embodied Agents (GENEA) Challenge 2023. Our proposed model, DiffuseStyleGesture+, leverages a diffusion model to generate gestures automatically. It incorporates a variety of modalities, including audio, text, speaker ID, and seed gestures.
  arXiv  Detail & Related papers  (2023-08-26T13:34:17Z)
• Textually Pretrained Speech Language Models [107.10344535390956]
  We propose TWIST, a method for training SpeechLMs using a warm-start from a pretrained textual language models. We show using both automatic and human evaluations that TWIST outperforms a cold-start SpeechLM across the board.
  arXiv  Detail & Related papers  (2023-05-22T13:12:16Z)
• NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality [123.97136358092585]
  We develop a TTS system called NaturalSpeech that achieves human-level quality on a benchmark dataset. Specifically, we leverage a variational autoencoder (VAE) for end-to-end text to waveform generation. Experiment evaluations on popular LJSpeech dataset show that our proposed NaturalSpeech achieves -0.01 CMOS to human recordings at the sentence level.
  arXiv  Detail & Related papers  (2022-05-09T16:57:35Z)
• LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech [67.88748572167309]
  We present LDNet, a unified framework for mean opinion score (MOS) prediction. We propose two inference methods that provide more stable results and efficient computation.
  arXiv  Detail & Related papers  (2021-10-18T08:52:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.