Towards dialect-inclusive recognition in a low-resource language: are balanced corpora the answer?

• URL: http://arxiv.org/abs/2307.07295v1
• Date: Fri, 14 Jul 2023 12:18:38 GMT
• Title: Towards dialect-inclusive recognition in a low-resource language: are balanced corpora the answer?
• Authors: Liam Lonergan, Mengjie Qian, Neasa N\'i Chiar\'ain, Christer Gobl, Ailbhe N\'i Chasaide
• Abstract summary: This study is a diagnostic to quantify the effect of the speaker's dialect on recognition performance. 12 ASR systems were trained using dialect-balanced training corpora and modified versions of the baseline corpora. Results indicate that dialect-balanced corpora do not yield a similar performance across the dialects. There is a close relationship between Co and Mu dialects, but one that is not symmetrical.
• Score: 5.1121440213561335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ASR systems are generally built for the spoken 'standard', and their performance declines for non-standard dialects/varieties. This is a problem for a language like Irish, where there is no single spoken standard, but rather three major dialects: Ulster (Ul), Connacht (Co) and Munster (Mu). As a diagnostic to quantify the effect of the speaker's dialect on recognition performance, 12 ASR systems were trained, firstly using baseline dialect-balanced training corpora, and then using modified versions of the baseline corpora, where dialect-specific materials were either subtracted or added. Results indicate that dialect-balanced corpora do not yield a similar performance across the dialects: the Ul dialect consistently underperforms, whereas Mu yields lowest WERs. There is a close relationship between Co and Mu dialects, but one that is not symmetrical. These results will guide future corpus collection and system building strategies to optimise for cross-dialect performance equity.

Related papers

• Voices Unheard: NLP Resources and Models for Yorùbá Regional Dialects [72.18753241750964]
  Yorub'a is an African language with roughly 47 million speakers. Recent efforts to develop NLP technologies for African languages have focused on their standard dialects. We take steps towards bridging this gap by introducing a new high-quality parallel text and speech corpus.
  arXiv  Detail & Related papers  (2024-06-27T22:38:04Z)
• Task-Agnostic Low-Rank Adapters for Unseen English Dialects [52.88554155235167]
  Large Language Models (LLMs) are trained on corpora disproportionally weighted in favor of Standard American English. By disentangling dialect-specific and cross-dialectal information, HyperLoRA improves generalization to unseen dialects in a task-agnostic fashion.
  arXiv  Detail & Related papers  (2023-11-02T01:17:29Z)
• Towards spoken dialect identification of Irish [5.1121440213561335]
  The Irish language is rich in its diversity of dialects and accents. A recent study investigating dialect bias in Irish ASR found that performance for the Ulster dialect was consistently worse than for the Connacht or Munster dialects. The present experiments investigate spoken dialect identification of Irish, with a view to incorporating such a system into the speech recognition pipeline.
  arXiv  Detail & Related papers  (2023-07-14T16:03:09Z)
• DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules [64.93179829965072]
  DADA is a modular approach to imbue SAE-trained models with multi-dialectal robustness. We show that DADA is effective for both single task and instruction fine language models.
  arXiv  Detail & Related papers  (2023-05-22T18:43:31Z)
• Multi-VALUE: A Framework for Cross-Dialectal English NLP [49.55176102659081]
  Multi- Dialect is a controllable rule-based translation system spanning 50 English dialects. Stress tests reveal significant performance disparities for leading models on non-standard dialects. We partner with native speakers of Chicano and Indian English to release new gold-standard variants of the popular CoQA task.
  arXiv  Detail & Related papers  (2022-12-15T18:17:01Z)
• Dialect-robust Evaluation of Generated Text [40.85375247260744]
  We formalize dialect robustness and dialect awareness as goals for NLG evaluation metrics. Applying the suite to current state-of-the-art metrics, we demonstrate that they are not dialect-robust.
  arXiv  Detail & Related papers  (2022-11-02T07:12:23Z)
• A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognition [80.87085897419982]
  We propose a novel acoustic modeling technique for accurate multi-dialect speech recognition with a single AM. Our proposed AM is dynamically adapted based on both dialect information and its internal representation, which results in a highly adaptive AM for handling multiple dialects simultaneously. The experimental results on large scale speech datasets show that the proposed AM outperforms all the previous ones, reducing word error rates (WERs) by 8.11% relative compared to a single all-dialects AM and by 7.31% relative compared to dialect-specific AMs.
  arXiv  Detail & Related papers  (2022-05-06T06:07:09Z)
• VALUE: Understanding Dialect Disparity in NLU [50.35526025326337]
  We construct rules for 11 features of African American Vernacular English (AAVE) We recruit fluent AAVE speakers to validate each feature transformation via linguistic acceptability judgments. Experiments show that these new dialectal features can lead to a drop in model performance.
  arXiv  Detail & Related papers  (2022-04-06T18:30:56Z)
• The Norwegian Parliamentary Speech Corpus [0.5874142059884521]
  The Norwegian Parliamentary Speech Corpus (NPSC) is a speech dataset with recordings of meetings from Stortinget, the Norwegian parliament. It is the first, publicly available dataset containing unscripted, Norwegian speech designed for training of automatic speech recognition (ASR) systems. Training on the NPSC is shown to have a "democratizing" effect in terms of dialects, as improvements are generally larger for dialects with higher WER from the baseline system.
  arXiv  Detail & Related papers  (2022-01-26T11:41:55Z)
• Learning to Recognize Dialect Features [21.277962038423123]
  We introduce the task of dialect feature detection, and present two multitask learning approaches. We train our models on a small number of minimal pairs, building on how linguists typically define dialect features.
  arXiv  Detail & Related papers  (2020-10-23T23:25:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.