Residual-Guided Non-Intrusive Speech Quality Assessment

• URL: http://arxiv.org/abs/2203.11499v1
• Date: Tue, 22 Mar 2022 07:19:58 GMT
• Title: Residual-Guided Non-Intrusive Speech Quality Assessment
• Abstract summary: This paper proposes an approach to improve Non-Intrusive speech quality assessment based on the residuals between impaired speech and enhanced speech. We generate an enhanced speech on the impaired speech to compensate for the absence of the reference audio, then pair the information of residuals with the impaired speech.
• Score: 4.7590051176368915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes an approach to improve Non-Intrusive speech quality assessment(NI-SQA) based on the residuals between impaired speech and enhanced speech. The difficulty in our task is particularly lack of information, for which the corresponding reference speech is absent. We generate an enhanced speech on the impaired speech to compensate for the absence of the reference audio, then pair the information of residuals with the impaired speech. Compared to feeding the impaired speech directly into the model, residuals could bring some extra helpful information from the contrast in enhancement. The human ear is sensitive to certain noises but different to deep learning model. Causing the Mean Opinion Score(MOS) the model predicted is not enough to fit our subjective sensitive well and causes deviation. These residuals have a close relationship to reference speech and then improve the ability of the deep learning models to predict MOS. During the training phase, experimental results demonstrate that paired with residuals can quickly obtain better evaluation indicators under the same conditions. Furthermore, our final results improved 31.3 percent and 14.1 percent, respectively, in PLCC and RMSE.

Related papers

• Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations [6.951409243358033]
  Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech research.<n>We investigate this via controlled perturbations on speech: acoustic degradation, prosodic errors, and manipulation of speaker-specific characteristics such as pitch and speaking rate.<n>Results show that most models track acoustic degradation well, while all are insensitive to prosodic errors despite large subjective score drops.<n>For speaker characteristics, models exhibit a double dissociation: strong mean fundamental frequency (F0) biases absent in human ratings, yet insensitivity to speaking rate and F0 variability that humans notice.
  arXiv  Detail & Related papers  (2026-06-18T08:49:24Z)
• Confidence-Guided Error Correction for Disordered Speech Recognition [10.275737387265321]
  We investigate the use of large language models (LLMs) as post-processing modules for automatic speech recognition (ASR)<n>We propose confidence-informed prompting, where word-level uncertainty estimates are embedded directly into LLM training to improve robustness and generalization across speakers and datasets.<n>We fine-tune a LLaMA 3.1 model and compare our approach to both transcript-only fine-tuning and post hoc confidence-based filtering.
  arXiv  Detail & Related papers  (2025-09-29T17:00:38Z)
• HuBERT-VIC: Improving Noise-Robust Automatic Speech Recognition of Speech Foundation Model via Variance-Invariance-Covariance Regularization [7.751856268560216]
  We propose HuBERT-VIC, a noise-robust SFM with variance, in- variance, and covariance regularization objectives.<n>These objectives adjust the statistics of noisy speech representations, enabling the model to capture diverse acoustic characteristics.<n>When applied to HuBERT, our model shows relative performance improvements of 23.3% on LibriSpeech test-clean and 13.2% on test-other, compared to the baseline model pre-trained on noisy speech.
  arXiv  Detail & Related papers  (2025-08-17T08:54:25Z)
• Audio Large Language Models Can Be Descriptive Speech Quality Evaluators [46.765203628127345]
  We introduce the first natural language-based speech evaluation corpus, generated from authentic human ratings. This corpus offers detailed analysis across multiple dimensions and identifies causes of quality degradation. We propose an alignment approach with LLM distillation (ALLD) to guide the audio LLM in extracting relevant information from raw speech.
  arXiv  Detail & Related papers  (2025-01-27T22:47:51Z)
• Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speech [18.10421212423484]
  We leverage an unconditional diffusion model trained only on clean speech for the assessment of speech quality. We show that the quality of a speech utterance can be assessed by estimating the likelihood of a corresponding sample in the terminating Gaussian distribution. Our proposed log-likelihoods show promising results, correlating well with intrusive speech quality metrics such as POLQA and SI-SDR.
  arXiv  Detail & Related papers  (2024-10-23T12:53:58Z)
• Ontology-aware Learning and Evaluation for Audio Tagging [56.59107110017436]
  Mean average precision (mAP) metric treats different kinds of sound as independent classes without considering their relations. Ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation. We conduct human evaluations and demonstrate that OmAP is more consistent with human perception than mAP.
  arXiv  Detail & Related papers  (2022-11-22T11:35:14Z)
• Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation [6.382013662443799]
  Estimating dimensional emotions from acoustic speech signals is challenging. We show that pre-trained acoustic models can improve valence estimation from speech. We report new state-of-the-art "text-free" acoustic-only dimensional emotion estimation.
  arXiv  Detail & Related papers  (2022-07-02T17:34:44Z)
• MOSRA: Joint Mean Opinion Score and Room Acoustics Speech Quality Assessment [12.144133923535714]
  This paper presents MOSRA: a non-intrusive multi-dimensional speech quality metric. It can predict room acoustics parameters alongside the overall mean opinion score (MOS) for speech quality. We also show that this joint training method enhances the blind estimation of room acoustics.
  arXiv  Detail & Related papers  (2022-04-04T09:38:15Z)
• Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation [59.41186714127256]
  Dysarthric speech reconstruction (DSR) aims to improve the quality of dysarthric speech. Speaker encoder (SE) optimized for speaker verification has been explored to control the speaker identity. We propose a novel multi-task learning strategy, i.e., adversarial speaker adaptation (ASA)
  arXiv  Detail & Related papers  (2022-02-18T08:59:36Z)
• Investigation of Data Augmentation Techniques for Disordered Speech Recognition [69.50670302435174]
  This paper investigates a set of data augmentation techniques for disordered speech recognition. Both normal and disordered speech were exploited in the augmentation process. The final speaker adapted system constructed using the UASpeech corpus and the best augmentation approach based on speed perturbation produced up to 2.92% absolute word error rate (WER)
  arXiv  Detail & Related papers  (2022-01-14T17:09:22Z)
• Self-Supervised Learning for speech recognition with Intermediate layer supervision [52.93758711230248]
  We propose Intermediate Layer Supervision for Self-Supervised Learning (ILS-SSL) ILS-SSL forces the model to concentrate on content information as much as possible by adding an additional SSL loss on the intermediate layers. Experiments on LibriSpeech test-other set show that our method outperforms HuBERT significantly.
  arXiv  Detail & Related papers  (2021-12-16T10:45:05Z)
• Self-Normalized Importance Sampling for Neural Language Modeling [97.96857871187052]
  In this work, we propose self-normalized importance sampling. Compared to our previous work, the criteria considered in this work are self-normalized and there is no need to further conduct a correction step. We show that our proposed self-normalized importance sampling is competitive in both research-oriented and production-oriented automatic speech recognition tasks.
  arXiv  Detail & Related papers  (2021-11-11T16:57:53Z)
• Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning [57.4036085386653]
  We show that prompt-based models for sentence pair classification tasks still suffer from a common pitfall of adopting inferences based on lexical overlap. We then show that adding a regularization that preserves pretraining weights is effective in mitigating this destructive tendency of few-shot finetuning.
  arXiv  Detail & Related papers  (2021-09-09T10:10:29Z)
• Improved Robustness to Disfluencies in RNN-Transducer Based Speech Recognition [1.8702587873591643]
  We investigate data selection and preparation choices aiming for improved robustness of RNN-T ASR to speech disfluencies. We show that after including a small amount of data with disfluencies in the training set the recognition accuracy on the tests with disfluencies and stuttering improves.
  arXiv  Detail & Related papers  (2020-12-11T11:47:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.