Calibration-Reasoning Framework for Descriptive Speech Quality Assessment

• URL: http://arxiv.org/abs/2603.10175v1
• Date: Tue, 10 Mar 2026 19:14:17 GMT
• Title: Calibration-Reasoning Framework for Descriptive Speech Quality Assessment
• Abstract summary: We introduce a novel post-training method that tailors the foundational Audio Large Language Model for multidimensional reasoning.<n>We reach state-of-the-art results of 0.71 mean PCC score on the multidimensional QualiSpeech benchmark.<n>Our fine-grained GRPO rewards substantially advance the model's ability to pinpoint and classify audio artifacts in time.
• Score: 55.51206474234095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explainable speech quality assessment requires moving beyond Mean Opinion Scores (MOS) to analyze underlying perceptual dimensions. To address this, we introduce a novel post-training method that tailors the foundational Audio Large Language Model for multidimensional reasoning, detection and classification of audio artifacts. First, a calibration stage aligns the model to predict predefined perceptual dimensions. Second, a reinforcement learning stage leverages Group Relative Policy Optimization (GRPO) with dimension-specific rewards to heavily enhance accuracy of descriptions and temporal localization of quality issues. With this approach we reach state-of-the-art results of 0.71 mean PCC score on the multidimensional QualiSpeech benchmark and 13% improvement in MOS prediction driven by RL-based reasoning. Furthermore, our fine-grained GRPO rewards substantially advance the model's ability to pinpoint and classify audio artifacts in time.

Related papers

• DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization [17.0840895033126]
  Speech quality assessment aims to predict Mean Opinion Scores consistent with human subjective perception.<n>We introduce explicit distortion localization as auxiliary knowledge for speech quality assessment.<n>We propose DAMOS, a distortion-aware speech quality assessment framework.
  arXiv  Detail & Related papers  (2026-08-21T14:46:27Z)
• SpeechQualityLLM: LLM-Based Multimodal Assessment of Speech Quality [2.1178416840822027]
  Speech quality assessment is central to telephony, VoIP, and streaming systems, where large volumes of degraded audio must be monitored and optimized at scale.<n>We introduce SpeechQualityLLM, a multimodal speech quality question-answering (QA) system that couples an audio encoder with a language model and is trained on the NISQA corpus using template-based question-answer pairs.<n>Our system is supervised to generate textual answers from which numeric predictions are parsed and evaluated with standard regression and ranking metrics.
  arXiv  Detail & Related papers  (2025-12-09T04:39:50Z)
• Thinking While Listening: Simple Test Time Scaling For Audio Classification [61.3564313676731]
  We propose a framework that enables neural models to "think while listening" to everyday sounds, thereby enhancing audio classification performance.<n>Motivated by recent advances in the reasoning capabilities of large language models, we address two central questions: (i) how can thinking be incorporated into existing audio classification pipelines to enable reasoning in the category space and improve performance, and (ii) can a new architecture be designed from the ground up to support both thinking and test-time scaling.
  arXiv  Detail & Related papers  (2025-09-24T01:17:24Z)
• APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech [20.311467577180274]
  Speech quality assessment aims to quantify subjective human perception of speech.<n>We propose an auditory perception guided-MOS prediction model (APG-MOS)<n>APG-MOS integrates auditory modeling with semantic analysis to enhance consistency with human judgments.
  arXiv  Detail & Related papers  (2025-04-29T05:45:09Z)
• $C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction [80.57232374640911]
  We propose a model-agnostic strategy called the Mask-And-Recover (MAR)<n>MAR integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules.<n>To better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model.
  arXiv  Detail & Related papers  (2025-04-01T13:01:30Z)
• Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training [68.94373533768501]
  We model knowledge retention, the capacity of a pre-trained language model to memorize factual information from its corpus, and introduce a principled method to estimate it prior to training.<n>We propose Size-dependent Mutual Information (SMI), an information-theoretic predictor that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book question answering (QA) accuracy.
  arXiv  Detail & Related papers  (2025-02-06T13:23:53Z)
• Audio Large Language Models Can Be Descriptive Speech Quality Evaluators [46.765203628127345]
  We introduce the first natural language-based speech evaluation corpus, generated from authentic human ratings.<n>This corpus offers detailed analysis across multiple dimensions and identifies causes of quality degradation.<n>We propose an alignment approach with LLM distillation (ALLD) to guide the audio LLM in extracting relevant information from raw speech.
  arXiv  Detail & Related papers  (2025-01-27T22:47:51Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors.<n>We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models.<n>In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• Ontology-aware Learning and Evaluation for Audio Tagging [56.59107110017436]
  Mean average precision (mAP) metric treats different kinds of sound as independent classes without considering their relations. Ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation. We conduct human evaluations and demonstrate that OmAP is more consistent with human perception than mAP.
  arXiv  Detail & Related papers  (2022-11-22T11:35:14Z)
• Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation [6.382013662443799]
  Estimating dimensional emotions from acoustic speech signals is challenging. We show that pre-trained acoustic models can improve valence estimation from speech. We report new state-of-the-art "text-free" acoustic-only dimensional emotion estimation.
  arXiv  Detail & Related papers  (2022-07-02T17:34:44Z)
• Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features [31.59528815233441]
  We propose a cross-domain multi-objective speech assessment model called MOSA-Net, which can estimate multiple speech assessment metrics simultaneously.<n> Experimental results show that MOSA-Net can improve the linear correlation coefficient (LCC) by 0.026 (0.990 vs 0.964 in seen noise environments) and 0.012 (0.969 vs 0.957 in unseen noise environments) in perceptual evaluation of speech quality (PESQ) prediction.
  arXiv  Detail & Related papers  (2021-11-03T17:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.