Multi-Task Learning for Non-Canonical Phoneme Recognition via Articulatory Feature Decomposition

• URL: http://arxiv.org/abs/2608.22273v1
• Date: Sun, 23 Aug 2026 08:03:18 GMT
• Title: Multi-Task Learning for Non-Canonical Phoneme Recognition via Articulatory Feature Decomposition
• Abstract summary: Pathological and more broadly non-canonical speech present significant challenges for automatic phoneme recognition.<n>Existing phoneme recognition systems are typically trained on canonical speech and treat phonemes as atomic categorical labels.<n>We introduce a linguistically structured approach to non-canonical phoneme recognition that decomposes phoneme prediction into articulatory feature dimensions.
• Score: 1.0737260344472548
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pathological and more broadly non-canonical speech present significant challenges for automatic phoneme recognition due to systematic deviations from canonical pronunciation and limited availability of labeled clinical speech data. Existing phoneme recognition systems are typically trained on canonical speech and treat phonemes as atomic categorical labels, limiting their ability to detect structured articulatory errors common in speech disorders and accents. In this work, we introduce a linguistically structured approach to non-canonical phoneme recognition that decomposes phoneme prediction into articulatory feature dimensions such as manner, place, and voicing. We implement this formulation using a hierarchical multi-task learning architecture in which task-specific articulatory feature heads learn feature-level representations that are subsequently integrated through a cross-attention-based fusion module to produce phoneme predictions. To address the scarcity and noise of pathological speech labels, we combine this framework with semi-supervised learning via Momentum Pseudo-Labeling (MPL) and propose a cascaded training strategy that progressively introduces articulatory feature tasks while employing staged unfreezing of a pretrained speech encoder. Experiments on L2-ARCTIC, used as a proxy for pathological speech variation, show that the proposed approach achieves substantial improvements in phoneme recognition performance compared to strong baseline architectures, while yielding interpretable error patterns aligned with phonological feature structure. These results suggest that articulatory feature supervision is a promising strategy for robust and interpretable phoneme recognition in non-canonical speech, and motivate future validation on clinically diagnosed pathological speech datasets.

Related papers

• INSPIRE: A Benchmark for Instruction-Aware Speech Retrieval [53.79335341504602]
  INSPIRE is the first benchmark for instruction-aware speech retrieval.<n>We evaluate four retrieval paradigms: large audio-language models, cascaded pipelines, self-supervised speech models, and contrastive audio-language models.
  arXiv  Detail & Related papers  (2026-08-17T07:35:18Z)
• Speech Codec Probing from Semantic and Phonetic Perspectives [49.01048570474675]
  Speech tokenizers are essential for connecting speech to large language models (LLMs) in multimodal systems.<n> emerging evidence suggests that what is termed "semantic" in speech representations does not align with text-derived semantics.
  arXiv  Detail & Related papers  (2026-03-11T03:32:25Z)
• Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modeling [22.333214778384487]
  The prevalence of neurological disorders associated with dysarthria motivates the need for automated intelligibility assessment methods.<n>We present a multilingual phoneme-production assessment framework that integrates universal phone recognition with language-specific phoneme interpretation.
  arXiv  Detail & Related papers  (2026-01-29T03:12:11Z)
• Towards Inclusive Communication: A Unified Framework for Generating Spoken Language from Sign, Lip, and Audio [52.859261069569165]
  We propose the first unified framework capable of handling diverse combinations of sign language, lip movements, and audio for spoken-language text generation.<n>We focus on three main objectives: (i) designing a unified, modality-agnostic architecture capable of effectively processing heterogeneous inputs; (ii) exploring the underexamined synergy among modalities, particularly the role of lip movements as non-manual cues in sign language comprehension; and (iii) achieving performance on par with or better than state-of-the-art models specialized for individual tasks.
  arXiv  Detail & Related papers  (2025-08-28T06:51:42Z)
• Languages in Multilingual Speech Foundation Models Align Both Phonetically and Semantically [58.019484208091534]
  Cross-lingual alignment in pretrained language models (LMs) has enabled efficient transfer in text-based LMs.<n>It remains an open question whether findings and methods from text-based cross-lingual alignment apply to speech.
  arXiv  Detail & Related papers  (2025-05-26T07:21:20Z)
• Anomaly Detection and Localization for Speech Deepfakes via Feature Pyramid Matching [8.466707742593078]
  Speech deepfakes are synthetic audio signals that can imitate target speakers' voices.<n>Existing methods for detecting speech deepfakes rely on supervised learning.<n>We introduce a novel interpretable one-class detection framework, which reframes speech deepfake detection as an anomaly detection task.
  arXiv  Detail & Related papers  (2025-03-23T11:15:22Z)
• DyPCL: Dynamic Phoneme-level Contrastive Learning for Dysarthric Speech Recognition [12.923409319624254]
  We propose a Dynamic Phoneme-level Contrastive Learning (DyPCL) method to bridge dysarthric speech recognition gaps.<n>We decompose the speech utterance into phoneme segments for phoneme-level contrastive learning, leveraging dynamic connectionist temporal classification alignment.<n>Our approach to training by difficulty levels alleviates the inherent variability of speakers, better identifying challenging speeches.
  arXiv  Detail & Related papers  (2025-01-31T10:25:42Z)
• Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition [48.56414496900755]
  This work uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully.
  arXiv  Detail & Related papers  (2022-04-01T14:25:19Z)
• Unsupervised Domain Adaptation in Speech Recognition using Phonetic Features [6.872447420442981]
  We propose a technique to perform unsupervised gender-based domain adaptation in speech recognition using phonetic features. Experiments are performed on the TIMIT dataset and there is a considerable decrease in the phoneme error rate using the proposed approach.
  arXiv  Detail & Related papers  (2021-08-04T06:22:12Z)
• Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification [81.80311855996584]
  We propose a novel intent classification framework that employs acoustic features extracted from a pretrained speech recognition system and linguistic features learned from a pretrained language model. We achieve 90.86% and 99.07% accuracy on ATIS and Fluent speech corpus, respectively.
  arXiv  Detail & Related papers  (2021-02-15T07:20:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.