Model Adaptation for ASR in low-resource Indian Languages

• URL: http://arxiv.org/abs/2307.07948v1
• Date: Sun, 16 Jul 2023 05:25:51 GMT
• Title: Model Adaptation for ASR in low-resource Indian Languages
• Authors: Abhayjeet Singh, Arjun Singh Mehta, Ashish Khuraishi K S, Deekshitha G, Gauri Date, Jai Nanavati, Jesuraja Bandekar, Karnalius Basumatary, Karthika P, Sandhya Badiger, Sathvik Udupa, Saurabh Kumar, Savitha, Prasanta Kumar Ghosh, Prashanthi V, Priyanka Pai, Raoul Nanavati, Rohan Saxena, Sai Praneeth Reddy Mora, Srinivasa Raghavan
• Abstract summary: Automatic speech recognition (ASR) performance has improved drastically in recent years, mainly enabled by self-supervised learning (SSL) based acoustic models like wav2vec2 and large-scale multi-lingual training like Whisper. A huge challenge still exists for low-resource languages where the availability of both audio and text is limited. This is where a lot of adaptation and fine-tuning techniques can be applied to overcome the low-resource nature of the data by utilising well-resourced similar languages. It could be the case that an abundance of acoustic data in a language reduces the need for large text-only corpora
• Score: 28.02064068964355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic speech recognition (ASR) performance has improved drastically in recent years, mainly enabled by self-supervised learning (SSL) based acoustic models such as wav2vec2 and large-scale multi-lingual training like Whisper. A huge challenge still exists for low-resource languages where the availability of both audio and text is limited. This is further complicated by the presence of multiple dialects like in Indian languages. However, many Indian languages can be grouped into the same families and share the same script and grammatical structure. This is where a lot of adaptation and fine-tuning techniques can be applied to overcome the low-resource nature of the data by utilising well-resourced similar languages. In such scenarios, it is important to understand the extent to which each modality, like acoustics and text, is important in building a reliable ASR. It could be the case that an abundance of acoustic data in a language reduces the need for large text-only corpora. Or, due to the availability of various pretrained acoustic models, the vice-versa could also be true. In this proposed special session, we encourage the community to explore these ideas with the data in two low-resource Indian languages of Bengali and Bhojpuri. These approaches are not limited to Indian languages, the solutions are potentially applicable to various languages spoken around the world.

Related papers

• Enabling ASR for Low-Resource Languages: A Comprehensive Dataset Creation Approach [0.6445605125467574]
  This study introduces a novel pipeline designed to generate ASR training datasets from audiobooks. The common structure of these audiobooks poses a unique challenge due to the extensive length of audio segments. We propose a method for effectively aligning audio with its corresponding text and segmenting it into lengths suitable for ASR training.
  arXiv  Detail & Related papers  (2024-06-03T15:38:40Z)
• Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon [78.12363425794214]
  We focus on zero-shot sentiment analysis tasks across 34 languages, including 6 high/medium-resource languages, 25 low-resource languages, and 3 code-switching datasets. We demonstrate that pretraining using multilingual lexicons, without using any sentence-level sentiment data, achieves superior zero-shot performance compared to models fine-tuned on English sentiment datasets.
  arXiv  Detail & Related papers  (2024-02-03T10:41:05Z)
• Lip Reading for Low-resource Languages by Learning and Combining General Speech Knowledge and Language-specific Knowledge [57.38948190611797]
  This paper proposes a novel lip reading framework, especially for low-resource languages. Since low-resource languages do not have enough video-text paired data to train the model, it is regarded as challenging to develop lip reading models for low-resource languages.
  arXiv  Detail & Related papers  (2023-08-18T05:19:03Z)
• Language-universal phonetic encoder for low-resource speech recognition [28.21805271848413]
  We leverage International Phonetic Alphabet (IPA) based language-universal phonetic model to improve low-resource ASR performances. Our approach and adaptation are effective on extremely low-resource languages, even within domain- and language-mismatched scenarios.
  arXiv  Detail & Related papers  (2023-05-19T10:24:30Z)
• Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples [89.16814518860357]
  The objective of this work is to explore the learning of visually grounded speech models (VGS) from multilingual perspective. Our key contribution in this work is to leverage the power of a high-resource language in a bilingual visually grounded speech model to improve the performance of a low-resource language.
  arXiv  Detail & Related papers  (2023-03-30T16:34:10Z)
• A Survey of Multilingual Models for Automatic Speech Recognition [6.657361001202456]
  Cross-lingual transfer is an attractive solution to the problem of multilingual Automatic Speech Recognition. Recent advances in Self Supervised Learning are opening up avenues for unlabeled speech data to be used in multilingual ASR models. We present best practices for building multilingual models from research across diverse languages and techniques.
  arXiv  Detail & Related papers  (2022-02-25T09:31:40Z)
• Towards Building ASR Systems for the Next Billion Users [15.867823754118422]
  We make contributions towards building ASR systems for low resource languages from the Indian subcontinent. First, we curate 17,000 hours of raw speech data for 40 Indian languages. Using this raw speech data we pretrain several variants of wav2vec style models for 40 Indian languages.
  arXiv  Detail & Related papers  (2021-11-06T19:34:33Z)
• Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity [81.51206991542242]
  Cross-lingual transfer offers a compelling way to help bridge this digital divide. Current cross-lingual algorithms have shown success in text-based tasks and speech-related tasks over some low-resource languages. We propose a language similarity approach that can efficiently identify acoustic cross-lingual transfer pairs across hundreds of languages.
  arXiv  Detail & Related papers  (2021-11-02T01:55:17Z)
• Multilingual and code-switching ASR challenges for low resource Indian languages [59.2906853285309]
  We focus on building multilingual and code-switching ASR systems through two different subtasks related to a total of seven Indian languages. We provide a total of 600 hours of transcribed speech data, comprising train and test sets, in these languages. We also provide a baseline recipe for both the tasks with a WER of 30.73% and 32.45% on the test sets of multilingual and code-switching subtasks, respectively.
  arXiv  Detail & Related papers  (2021-04-01T03:37:01Z)
• That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages [72.9927937955371]
  We use the resources existing in other languages to train a multilingual automatic speech recognition model. We observe significant improvements across all languages in the multilingual setting, and stark degradation in the crosslingual setting. Our analysis uncovered that even the phones that are unique to a single language can benefit greatly from adding training data from other languages.
  arXiv  Detail & Related papers  (2020-05-16T22:28:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.