Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations

• URL: http://arxiv.org/abs/2606.19951v1
• Date: Thu, 18 Jun 2026 08:49:24 GMT
• Title: Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations
• Abstract summary: Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech research.<n>We investigate this via controlled perturbations on speech: acoustic degradation, prosodic errors, and manipulation of speaker-specific characteristics such as pitch and speaking rate.<n>Results show that most models track acoustic degradation well, while all are insensitive to prosodic errors despite large subjective score drops.<n>For speaker characteristics, models exhibit a double dissociation: strong mean fundamental frequency (F0) biases absent in human ratings, yet insensitivity to speaking rate and F0 variability that humans notice.
• Score: 6.951409243358033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech (TTS) research, yet their ability to capture quality differences beyond acoustic fidelity remains unclear. We investigate this via controlled perturbations on speech: acoustic degradation, prosodic errors, and manipulation of speaker-specific characteristics such as pitch and speaking rate. We obtained MOS predictions for these speech samples from both human listeners and the model, and analyzed the differences in their perceptual characteristics. Results show that most models track acoustic degradation well, while all are insensitive to prosodic errors despite large subjective score drops. For speaker characteristics, models exhibit a double dissociation: strong mean fundamental frequency (F0) biases absent in human ratings, yet insensitivity to speaking rate and F0 variability that humans notice. These findings highlight limitations of scalar MOS prediction beyond acoustic fidelity.

Related papers

• Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models [54.041320081289996]
  We present the first systematic empirical study of uncertainty estimation in audio-aware large language models (ALLMs)<n>We benchmark five representative methods, including predictive entropy, length-normalized entropy, semantic entropy, discrete semantic entropy, and P(True)<n>Our results reveal two key findings. First, semantic-level and verification-based methods consistently outperform token-level baselines on general audio reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-04-28T12:56:22Z)
• DEAF: A Benchmark for Diagnostic Evaluation of Acoustic Faithfulness in Audio Language Models [15.637236155997]
  Recent Audio Multimodal Large Language Models (Audio MLLMs) demonstrate impressive performance on speech benchmarks.<n>It remains unclear whether these models genuinely process acoustic signals or rely on text-based semantic inference.<n>We introduce DEAF, a benchmark of over 2,700 conflict stimuli spanning three acoustic dimensions.
  arXiv  Detail & Related papers  (2026-03-17T15:52:26Z)
• Investigation for Relative Voice Impression Estimation [16.024524623990484]
  This study investigates relative voice impression estimation (RIE)<n>RIE is a framework for predicting the perceptual difference between two utterances from the same speaker.<n>We compare three modeling approaches: classical acoustic features commonly used for speech emotion recognition, self-supervised speech representations, and multimodal large language models.
  arXiv  Detail & Related papers  (2026-02-15T14:54:52Z)
• Audio Large Language Models Can Be Descriptive Speech Quality Evaluators [46.765203628127345]
  We introduce the first natural language-based speech evaluation corpus, generated from authentic human ratings.<n>This corpus offers detailed analysis across multiple dimensions and identifies causes of quality degradation.<n>We propose an alignment approach with LLM distillation (ALLD) to guide the audio LLM in extracting relevant information from raw speech.
  arXiv  Detail & Related papers  (2025-01-27T22:47:51Z)
• Investigating the Impact of Model Instability on Explanations and Uncertainty [43.254616360807496]
  We simulate uncertainty in text input by introducing noise at inference time. We find that high uncertainty doesn't necessarily imply low explanation plausibility. This suggests that noise-augmented models may be better at identifying salient tokens when uncertain.
  arXiv  Detail & Related papers  (2024-02-20T13:41:21Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• Analysing the Impact of Audio Quality on the Use of Naturalistic Long-Form Recordings for Infant-Directed Speech Research [62.997667081978825]
  Modelling of early language acquisition aims to understand how infants bootstrap their language skills. Recent developments have enabled the use of more naturalistic training data for computational models. It is currently unclear how the sound quality could affect analyses and modelling experiments conducted on such data.
  arXiv  Detail & Related papers  (2023-05-03T08:25:37Z)
• Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis [6.382013662443799]
  We used multi-modal fusion representations from pre-trained models to generate state-of-the-art speech emotion estimation. We discovered that lexical representations are more robust to distortions compared to acoustic representations.
  arXiv  Detail & Related papers  (2023-03-03T18:22:32Z)
• Pre-Finetuning for Few-Shot Emotional Speech Recognition [20.894029832911617]
  We view speaker adaptation as a few-shot learning problem. We propose pre-finetuning speech models on difficult tasks to distill knowledge into few-shot downstream classification objectives.
  arXiv  Detail & Related papers  (2023-02-24T22:38:54Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)
• SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis [50.236929707024245]
  The SOMOS dataset is the first large-scale mean opinion scores (MOS) dataset consisting of solely neural text-to-speech (TTS) samples. It consists of 20K synthetic utterances of the LJ Speech voice, a public domain speech dataset.
  arXiv  Detail & Related papers  (2022-04-06T18:45:20Z)
• MOSRA: Joint Mean Opinion Score and Room Acoustics Speech Quality Assessment [12.144133923535714]
  This paper presents MOSRA: a non-intrusive multi-dimensional speech quality metric. It can predict room acoustics parameters alongside the overall mean opinion score (MOS) for speech quality. We also show that this joint training method enhances the blind estimation of room acoustics.
  arXiv  Detail & Related papers  (2022-04-04T09:38:15Z)
• Residual-Guided Non-Intrusive Speech Quality Assessment [4.7590051176368915]
  This paper proposes an approach to improve Non-Intrusive speech quality assessment based on the residuals between impaired speech and enhanced speech. We generate an enhanced speech on the impaired speech to compensate for the absence of the reference audio, then pair the information of residuals with the impaired speech.
  arXiv  Detail & Related papers  (2022-03-22T07:19:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.