Scaling A Simple Approach to Zero-Shot Speech Recognition

• URL: http://arxiv.org/abs/2407.17852v1
• Date: Thu, 25 Jul 2024 08:08:55 GMT
• Title: Scaling A Simple Approach to Zero-Shot Speech Recognition
• Abstract summary: MMS Zero-shot reduces the average character error rate by a relative 46% over 100 unseen languages. Our approach is only 2.5x higher compared to in-domain supervised baselines.
• Score: 45.83866201822545
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite rapid progress in increasing the language coverage of automatic speech recognition, the field is still far from covering all languages with a known writing script. Recent work showed promising results with a zero-shot approach requiring only a small amount of text data, however, accuracy heavily depends on the quality of the used phonemizer which is often weak for unseen languages. In this paper, we present MMS Zero-shot a conceptually simpler approach based on romanization and an acoustic model trained on data in 1,078 different languages or three orders of magnitude more than prior art. MMS Zero-shot reduces the average character error rate by a relative 46% over 100 unseen languages compared to the best previous work. Moreover, the error rate of our approach is only 2.5x higher compared to in-domain supervised baselines, while our approach uses no labeled data for the evaluation languages at all.

Related papers

• An Empirical Recipe for Universal Phone Recognition [63.45609714127985]
  Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks.<n>We present PhoneticXEUS -- trained on large-scale multilingual data.<n>We achieve state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER)
  arXiv  Detail & Related papers  (2026-03-30T22:12:48Z)
• Improving Multilingual ASR in the Wild Using Simple N-best Re-ranking [68.77659513993507]
  We present a simple and effective N-best re-ranking approach to improve multilingual ASR accuracy. Our results show spoken language identification accuracy improvements of 8.7% and 6.1%, respectively, and word error rates which are 3.3% and 2.0% lower on these benchmarks.
  arXiv  Detail & Related papers  (2024-09-27T03:31:32Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors. We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models. In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language [53.8622516025736]
  We propose a novel end-to-end zero-shot dialogue generation model ChatZero based on cross-lingual code-switching method. Experiments on the multilingual DailyDialog and DSTC7-AVSD datasets demonstrate that ChatZero can achieve more than 90% of the original performance.
  arXiv  Detail & Related papers  (2024-08-16T13:11:53Z)
• Initial Decoding with Minimally Augmented Language Model for Improved Lattice Rescoring in Low Resource ASR [0.532018200832244]
  This paper addresses the problem of improving speech recognition accuracy with lattice rescoring in low-resource languages. We minimally augment the baseline language model with word unigram counts that are present in a larger text corpus of the target language but absent in the baseline. We obtain 21.8% (Telugu) and 41.8% (Kannada) relative word error reduction with our proposed method.
  arXiv  Detail & Related papers  (2024-03-16T14:34:31Z)
• Multilingual Zero Resource Speech Recognition Base on Self-Supervise Pre-Trained Acoustic Models [14.887781621924255]
  This paper is the first attempt to extend the use of pre-trained models into word-level zero-resource speech recognition. It is done by fine-tuning the pre-trained models on IPA phoneme transcriptions and decoding with a language model trained on extra texts. Experiments on Wav2vec 2.0 and HuBERT models show that this method can achieve less than 20% word error rate on some languages.
  arXiv  Detail & Related papers  (2022-10-13T12:11:18Z)
• Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation [0.0]
  We modify the input tokens at both the encoder and decoder to include signals for the source and target languages. We show a performance gain when training from scratch, or finetuning a pretrained model with the proposed setup.
  arXiv  Detail & Related papers  (2022-08-11T14:42:42Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues [7.8378818005171125]
  Given a large-scale dialogue data set in one language, we can automatically produce an effective semantic for other languages using machine translation. We propose automatic translation of dialogue datasets with alignment to ensure faithful translation of slot values. We show that the succinct representation reduces the compounding effect of translation errors.
  arXiv  Detail & Related papers  (2021-11-04T01:08:14Z)
• AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages [75.08199398141744]
  We present AmericasNLI, an extension of XNLI (Conneau et al.), to 10 indigenous languages of the Americas. We conduct experiments with XLM-R, testing multiple zero-shot and translation-based approaches. We find that XLM-R's zero-shot performance is poor for all 10 languages, with an average performance of 38.62%.
  arXiv  Detail & Related papers  (2021-04-18T05:32:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.