Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment

• URL: http://arxiv.org/abs/2604.06191v1
• Date: Wed, 11 Mar 2026 04:10:35 GMT
• Title: Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment
• Abstract summary: Harf-Speech is a modular system scoring Arabic pronunciation at the phoneme level on a clinical scale.<n>Harf-Speech attains a Pearson correlation of 0.791 and ICC (2,1) of 0.659 with mean expert scores, outperforming existing end-to-end assessment frameworks.
• Score: 1.0898519359579681
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated phoneme-level pronunciation assessment is vital for scalable speech therapy and language learning, yet validated tools for Arabic remain scarce. We present Harf-Speech, a modular system scoring Arabic pronunciation at the phoneme level on a clinical scale. It combines an MSA phonetizer, a fine-tuned speech-to-phoneme model, Levenshtein alignment, and a blended scorer using longest common subsequence and edit-distance metrics. We fine-tune three ASR architectures on Arabic phoneme data and benchmark them with zero-shot multimodal models; the best, OmniASR-CTC-1B-v2, achieves 8.92\% phoneme error rate. Three certified speech-language pathologists independently scored 40 utterances for clinical validation. Harf-Speech attains a Pearson correlation of 0.791 and ICC(2,1) of 0.659 with mean expert scores, outperforming existing end-to-end assessment frameworks. These results show Harf-Speech yields clinically aligned, interpretable scores comparable to inter-rater expert agreement.

Related papers

• Neural Speaker Diarization via Multilingual Training: Evaluation on Low-Resource Nepali-Hindi Speech [0.0]
  This paper investigates speaker diarization for low-resource Nepali-Hindi speech through a multilingual training approach.<n>We compare two modern architectures: EEND with encoder-decoder attractors (EEND-EDA) and EEND with Perceiver-based attractors (DiaPer)
  arXiv  Detail & Related papers  (2026-06-21T12:10:45Z)
• Automated Pronunciation Evaluation for Korean Toddler Speech using Speech Diarization and Self-Supervised Learning [0.0]
  Speech sound disorders affect approximately 44% of Korean pediatric communication disorder cases.<n>This paper presents an end-to-end pipeline for automated pronunciation evaluation of Korean toddler speech.
  arXiv  Detail & Related papers  (2026-06-08T22:07:59Z)
• MUSCAT: MUltilingual, SCientific ConversATion Benchmark [60.87925076316812]
  The goal of multilingual speech technology is to facilitate seamless communication between individuals speaking different languages.<n>To create this experience, speech technology needs to address several challenges: Handling mixed multilingual input, specific vocabulary, and code-switching.<n>We propose a new benchmark to evaluate current Automatic Speech Recognition (ASR) systems, whether they are able to handle these challenges.
  arXiv  Detail & Related papers  (2026-04-17T10:39:01Z)
• Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modeling [22.333214778384487]
  The prevalence of neurological disorders associated with dysarthria motivates the need for automated intelligibility assessment methods.<n>We present a multilingual phoneme-production assessment framework that integrates universal phone recognition with language-specific phoneme interpretation.
  arXiv  Detail & Related papers  (2026-01-29T03:12:11Z)
• Enhancing Quranic Learning: A Multimodal Deep Learning Approach for Arabic Phoneme Recognition [0.0]
  This study proposes a transformer-based multimodal framework for Arabic phoneme mispronunciation detection.<n>The framework integrates UniSpeech-derived acoustic embeddings with BERT-based textual embeddings extracted from Whisper transcriptions.<n>The study contributes to the development of intelligent, speaker-independent, and multimodal Computer-Aided Language Learning (CALL) systems.
  arXiv  Detail & Related papers  (2025-11-21T18:25:46Z)
• Automatic Pronunciation Error Detection and Correction of the Holy Quran's Learners Using Deep Learning [0.0]
  We build a 98% automated pipeline to produce high-quality Quranic datasets.<n>We use our custom Quran Phonetic Script to encode Tajweed rules.<n>We release all code, data, and models as open-source.
  arXiv  Detail & Related papers  (2025-08-27T15:28:46Z)
• Towards stable AI systems for Evaluating Arabic Pronunciations [0.7999703756441757]
  We show that this phoneme-level task is challenging because isolated letters lack co-articulatory cues, provide no lexical context, and last only a few hundred milliseconds.<n>This study introduces a diverse, diacritised corpus of isolated Arabic letters and demonstrates that state-of-the-art wav2vec 2.0 models achieve only 35% accuracy on it.
  arXiv  Detail & Related papers  (2025-08-27T05:49:15Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors.<n>We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models.<n>In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• BLASER: A Text-Free Speech-to-Speech Translation Evaluation Metric [66.73705349465207]
  End-to-end speech-to-speech translation (S2ST) is generally evaluated with text-based metrics. We propose a text-free evaluation metric for end-to-end S2ST, named BLASER, to avoid the dependency on ASR systems.
  arXiv  Detail & Related papers  (2022-12-16T14:00:26Z)
• QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus [11.113497373432411]
  We introduce the largest transcribed Arabic speech corpus, QASR, collected from the broadcast domain. This multi-dialect speech dataset contains 2,000 hours of speech sampled at 16kHz crawled from Aljazeera news channel.
  arXiv  Detail & Related papers  (2021-06-24T13:20:40Z)
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data [54.733889961024445]
  We propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data. We evaluate the effectiveness of UniSpeech for cross-lingual representation learning on public CommonVoice corpus.
  arXiv  Detail & Related papers  (2021-01-19T12:53:43Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)
• Towards Zero-shot Learning for Automatic Phonemic Transcription [82.9910512414173]
  A more challenging problem is to build phonemic transcribers for languages with zero training data. Our model is able to recognize unseen phonemes in the target language without any training data. It achieves 7.7% better phoneme error rate on average over a standard multilingual model.
  arXiv  Detail & Related papers  (2020-02-26T20:38:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.