A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales

• URL: http://arxiv.org/abs/2606.09470v1
• Date: Mon, 08 Jun 2026 13:27:40 GMT
• Title: A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales
• Abstract summary: We propose a rubric-guided SpeechLLM for multi-aspect, multi-granular assessment.<n>The model jointly predicts ordinal labels at the sentence-level (accuracy, fluency, prosody), word/phoneme-level accuracy, and generates a natural-language rationale in the same response.
• Score: 8.300738063140129
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated L2 speech assessment can assign proficiency labels, but often lacks interpretability. We propose a rubric-guided SpeechLLM for multi-aspect, multi-granular assessment, trained with a hybrid objective combining supervised fine-tuning and Bounded Direct Preference Optimization. The model jointly predicts ordinal labels at the sentence-level (accuracy, fluency, prosody), word/phoneme-level accuracy, and generates a natural-language rationale in the same response. On SpeechOcean762, our approach matches or outperforms single-granularity models while remaining competitive with prior approaches. We analyze rationale reliability along two axes: self-consistency with model predictions and alignment with ground-truth labels, using sentiment consistency (plausibility) and mention-based agreement (faithfulness). Rationales are plausible at the sentence level, but faithfulness degrades at the word/phoneme level: references are sparse and weakly aligned with token-level labels.

Related papers

• Evaluation Pitfalls and Sparsity Limitations in LLM-based Confidence Estimates for Classification [1.5977080543655726]
  Common approaches such as verbalization produce extremely sparse outputs.<n>Qwen3-32B verbalizes only eight unique confidence values on SST-2, with over half being exactly 95%.<n>We address sparsity and the best AUARC (+2.3 points over vanilla verbalization) without incurring additional inference cost.
  arXiv  Detail & Related papers  (2026-08-05T14:24:34Z)
• Holistic Reliability Propagation: Decoupling Annotation and Prediction for Robust Noisy-Label [3.7874902461360627]
  Bilevel meta-learning produces two batch-normalized scalars per sample, alpha for the given label and beta for the pseudo-label, without constraining them to sum to one.<n>We use reliability-aware Mixup with global gating on the input branch and beta-gated pseudo-label positives on the contrastive branch.<n>On synthetic and real-world benchmarks, HRP improves average accuracy over strong baselines and remains competitive at the highest noise rates.
  arXiv  Detail & Related papers  (2026-05-20T05:24:37Z)
• Extending Minimal Pairs with Ordinal Surprisal Curves and Entropy Across Applied Domains [0.0]
  We extend surprisal-based evaluation from binary grammaticality to ordinal-scaled classification and scoring tasks.<n>Rather than asking models to generate answers, we measure the information-theoretic "surprise"<n>We explore this framework across four domains: social-ecological-technological systems classification, causal statement identification (binary and scaled), figurative language detection, and deductive qualitative coding.
  arXiv  Detail & Related papers  (2026-03-15T14:31:00Z)
• Rubric-Guided Fine-tuning of SpeechLLMs for Multi-Aspect, Multi-Rater L2 Reading-Speech Assessment [8.300738063140129]
  We introduce a rubric-guided reasoning framework that explicitly encodes multi-aspect human assessment criteria.<n>We develop an uncertainty-calibrated regression approach supported by conformal calibration for interpretable confidence intervals.<n>Our results demonstrate that rubric-guided, uncertainty-calibrated reasoning offers a principled path toward trustworthy and explainable SpeechLLM-based speech assessment.
  arXiv  Detail & Related papers  (2026-02-28T11:08:24Z)
• Judging with Confidence: Calibrating Autoraters to Preference Distributions [56.17041629492863]
  We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population.<n>We present two learning methods tailored to different data conditions.<n>Our results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution.
  arXiv  Detail & Related papers  (2025-09-30T20:36:41Z)
• Weakly-Supervised Contrastive Learning for Imprecise Class Labels [50.57424331797865]
  We introduce the concept of continuous semantic similarity'' to define positive and negative pairs.<n>We propose a graph-theoretic framework for weakly-supervised contrastive learning.<n>Our framework is highly versatile and can be applied to many weakly-supervised learning scenarios.
  arXiv  Detail & Related papers  (2025-05-28T06:50:40Z)
• Estimating Commonsense Plausibility through Semantic Shifts [66.06254418551737]
  We propose ComPaSS, a novel discriminative framework that quantifies commonsense plausibility by measuring semantic shifts.<n> Evaluations on two types of fine-grained commonsense plausibility estimation tasks show that ComPaSS consistently outperforms baselines.
  arXiv  Detail & Related papers  (2025-02-19T06:31:06Z)
• Localizing Factual Inconsistencies in Attributable Text Generation [74.11403803488643]
  We introduce QASemConsistency, a new formalism for localizing factual inconsistencies in attributable text generation.<n>We show that QASemConsistency yields factual consistency scores that correlate well with human judgments.
  arXiv  Detail & Related papers  (2024-10-09T22:53:48Z)
• Robust Representation Learning for Unreliable Partial Label Learning [86.909511808373]
  Partial Label Learning (PLL) is a type of weakly supervised learning where each training instance is assigned a set of candidate labels, but only one label is the ground-truth. This is known as Unreliable Partial Label Learning (UPLL) that introduces an additional complexity due to the inherent unreliability and ambiguity of partial labels. We propose the Unreliability-Robust Representation Learning framework (URRL) that leverages unreliability-robust contrastive learning to help the model fortify against unreliable partial labels effectively.
  arXiv  Detail & Related papers  (2023-08-31T13:37:28Z)
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation [49.295165476818866]
  This paper focuses on the unsupervised domain adaptation of transferring the knowledge from the source domain to the target domain in the context of semantic segmentation. Existing approaches usually regard the pseudo label as the ground truth to fully exploit the unlabeled target-domain data. This paper proposes to explicitly estimate the prediction uncertainty during training to rectify the pseudo label learning.
  arXiv  Detail & Related papers  (2020-03-08T12:37:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.