Language Complexity and Speech Recognition Accuracy: Orthographic Complexity Hurts, Phonological Complexity Doesn't

• URL: http://arxiv.org/abs/2406.09202v1
• Date: Thu, 13 Jun 2024 14:59:45 GMT
• Title: Language Complexity and Speech Recognition Accuracy: Orthographic Complexity Hurts, Phonological Complexity Doesn't
• Abstract summary: We investigate what linguistic factors affect the performance of Automatic Speech Recognition (ASR) models. We fine-tune the multilingual self-supervised pretrained model Wav2Vec2-XLSR-53 on 25 languages with 15 writing systems.
• Score: 9.944647907864256
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate what linguistic factors affect the performance of Automatic Speech Recognition (ASR) models. We hypothesize that orthographic and phonological complexities both degrade accuracy. To examine this, we fine-tune the multilingual self-supervised pretrained model Wav2Vec2-XLSR-53 on 25 languages with 15 writing systems, and we compare their ASR accuracy, number of graphemes, unigram grapheme entropy, logographicity (how much word/morpheme-level information is encoded in the writing system), and number of phonemes. The results demonstrate that orthographic complexities significantly correlate with low ASR accuracy, while phonological complexity shows no significant correlation.

Related papers

• Gated Multi-Graph Fusion via Graph Attention Networks for Alzheimer's Disease Detection [52.36198268369384]
  Spontaneous speech is a vital non-invasive biomarker for Alzheimer's Disease (AD)<n>We propose a Multi-View Gated Graph Attention Network that transcribes audio via Automatic Speech Recognition (ASR) to construct semantic, dependency, and co-occurrence graphs.
  arXiv  Detail & Related papers  (2026-06-30T06:15:59Z)
• Hard to Be Heard: Phoneme-Level ASR Analysis of Phonologically Complex, Low-Resource Endangered Languages [2.3513645401551337]
  We present a phoneme-level analysis of automatic speech recognition (ASR) for two low-resourced and phonologically complex East Caucasian languages.<n>Existing recordings and transcriptions are consolidated and processed into a form suitable for ASR training and evaluation.
  arXiv  Detail & Related papers  (2026-04-20T12:54:14Z)
• An Empirical Recipe for Universal Phone Recognition [63.45609714127985]
  Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks.<n>We present PhoneticXEUS -- trained on large-scale multilingual data.<n>We achieve state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER)
  arXiv  Detail & Related papers  (2026-03-30T22:12:48Z)
• The Degree of Language Diacriticity and Its Effect on Tasks [8.254230288283258]
  We propose a data-driven framework for quantifying diacritic complexity.<n>We compute these metrics over 24 corpora in 15 languages, spanning both single- and multi-diacritic scripts.<n>We find that across languages, higher diacritic complexity is strongly associated with lower restoration accuracy.
  arXiv  Detail & Related papers  (2026-03-29T12:04:38Z)
• Associative Syntax and Maximal Repetitions reveal context-dependent complexity in fruit bat communication [0.0]
  This study presents an unsupervised method to infer discreteness, syntax and temporal structures of fruit-bats vocalizations.<n>It evaluates the complexity of communication patterns in relation with behavioral context.
  arXiv  Detail & Related papers  (2025-11-30T19:01:59Z)
• Prominence-aware automatic speech recognition for conversational speech [5.908471365011944]
  This paper investigates prominence-aware automatic speech recognition.<n>It combines prominence detection and speech recognition for conversational Austrian German.<n>It shows that transformer-based models can effectively encode prosodic information.
  arXiv  Detail & Related papers  (2025-09-12T10:18:38Z)
• VALLR: Visual ASR Language Model for Lip Reading [28.561566996686484]
  Lip Reading, or Visual Automatic Speech Recognition, is a complex task requiring the interpretation of spoken language exclusively from visual cues. We propose a novel two-stage, phoneme-centric framework for Visual Automatic Speech Recognition (V-ASR) First, our model predicts a compact sequence of phonemes from visual inputs using a Video Transformer with a CTC head. This phoneme output then serves as the input to a fine-tuned Large Language Model (LLM), which reconstructs coherent words and sentences.
  arXiv  Detail & Related papers  (2025-03-27T11:52:08Z)
• Automatic Speech Recognition for Non-Native English: Accuracy and Disfluency Handling [0.0]
  This study assesses five cutting-edge ASR systems' recognition of non-native English accented speech using recordings from the L2-ARCTIC corpus. For read speech, Whisper and AssemblyAI achieved the best accuracy with mean Match Error Rates (MER) of 0.054 and 0.056 respectively. For spontaneous speech, RevAI performed best with a mean MER of 0.063.
  arXiv  Detail & Related papers  (2025-03-10T05:09:44Z)
• Towards Unsupervised Speech Recognition Without Pronunciation Models [57.222729245842054]
  In this article, we tackle the challenge of developing ASR systems without paired speech and text corpora. We experimentally demonstrate that an unsupervised speech recognizer can emerge from joint speech-to-speech and text-to-text masked token-infilling. This innovative model surpasses the performance of previous unsupervised ASR models under the lexicon-free setting.
  arXiv  Detail & Related papers  (2024-06-12T16:30:58Z)
• MulCogBench: A Multi-modal Cognitive Benchmark Dataset for Evaluating Chinese and English Computational Language Models [44.74364661212373]
  This paper proposes MulCogBench, a cognitive benchmark dataset collected from native Chinese and English participants. It encompasses a variety of cognitive data, including subjective semantic ratings, eye-tracking, functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) Results show that language models share significant similarities with human cognitive data and the similarity patterns are modulated by the data modality and stimuli complexity.
  arXiv  Detail & Related papers  (2024-03-02T07:49:57Z)
• Linguistic More: Taking a Further Step toward Efficient and Accurate Scene Text Recognition [92.6211155264297]
  Vision models have gained increasing attention due to their simplicity and efficiency in Scene Text Recognition (STR) task. Recent vision models suffer from two problems: (1) the pure vision-based query results in attention drift, which usually causes poor recognition and is summarized as linguistic insensitive drift (LID) problem in this paper. We propose a $textbfL$inguistic $textbfP$erception $textbfV$ision model (LPV) which explores the linguistic capability of vision model for accurate text recognition.
  arXiv  Detail & Related papers  (2023-05-09T02:52:47Z)
• Cross-Lingual Transfer of Cognitive Processing Complexity [11.939409227407769]
  We use sentence-level eye-tracking patterns as a cognitive indicator for structural complexity. We show that the multilingual model XLM-RoBERTa can successfully predict varied patterns for 13 typologically diverse languages.
  arXiv  Detail & Related papers  (2023-02-24T15:48:23Z)
• GraphQ IR: Unifying Semantic Parsing of Graph Query Language with Intermediate Representation [91.27083732371453]
  We propose a unified intermediate representation (IR) for graph query languages, namely GraphQ IR. With the IR's natural-language-like representation that bridges the semantic gap and its formally defined syntax that maintains the graph structure, neural semantic parsing can more effectively convert user queries into GraphQ IR. Our approach can consistently achieve state-of-the-art performance on KQA Pro, Overnight and MetaQA.
  arXiv  Detail & Related papers  (2022-05-24T13:59:53Z)
• Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition [48.56414496900755]
  This work uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully.
  arXiv  Detail & Related papers  (2022-04-01T14:25:19Z)
• Learning to pronounce as measuring cross lingual joint orthography-phonology complexity [0.0]
  We investigate what makes a language "hard to pronounce" by modelling the task of grapheme-to-phoneme (g2p) transliteration. We show that certain characteristics emerge that separate easier and harder languages with respect to learning to pronounce.
  arXiv  Detail & Related papers  (2022-01-29T14:44:39Z)
• Influence of ASR and Language Model on Alzheimer's Disease Detection [2.4698886064068555]
  We analyse the usage of a SotA ASR system to transcribe participant's spoken descriptions from a picture. We study the influence of a language model -- which tends to correct non-standard sequences of words -- with the lack of language model to decode the hypothesis from the ASR. The proposed system combines acoustic -- based on prosody and voice quality -- and lexical features based on the first occurrence of the most common words.
  arXiv  Detail & Related papers  (2021-09-20T10:41:39Z)
• LCP-RIT at SemEval-2021 Task 1: Exploring Linguistic Features for Lexical Complexity Prediction [4.86331990243181]
  This paper describes team LCP-RIT's submission to the SemEval-2021 Task 1: Lexical Complexity Prediction (LCP) Our system uses logistic regression and a wide range of linguistic features to predict the complexity of single words in this dataset. We evaluate the results in terms of mean absolute error, mean squared error, Pearson correlation, and Spearman correlation.
  arXiv  Detail & Related papers  (2021-05-18T18:55:04Z)
• Pareto Probing: Trading Off Accuracy for Complexity [87.09294772742737]
  We argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance. Our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations.
  arXiv  Detail & Related papers  (2020-10-05T17:27:31Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)
• Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction [80.38130122127882]
  We introduce 14 probing tasks targeting linguistic properties relevant to neural relation extraction (RE) We use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance.
  arXiv  Detail & Related papers  (2020-04-17T09:17:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.