That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages

• URL: http://arxiv.org/abs/2005.08118v1
• Date: Sat, 16 May 2020 22:28:09 GMT
• Title: That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages
• Authors: Piotr \.Zelasko, Laureano Moro-Vel\'azquez, Mark Hasegawa-Johnson, Odette Scharenborg, Najim Dehak
• Abstract summary: We use the resources existing in other languages to train a multilingual automatic speech recognition model. We observe significant improvements across all languages in the multilingual setting, and stark degradation in the crosslingual setting. Our analysis uncovered that even the phones that are unique to a single language can benefit greatly from adding training data from other languages.
• Score: 72.9927937955371
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Only a handful of the world's languages are abundant with the resources that enable practical applications of speech processing technologies. One of the methods to overcome this problem is to use the resources existing in other languages to train a multilingual automatic speech recognition (ASR) model, which, intuitively, should learn some universal phonetic representations. In this work, we focus on gaining a deeper understanding of how general these representations might be, and how individual phones are getting improved in a multilingual setting. To that end, we select a phonetically diverse set of languages, and perform a series of monolingual, multilingual and crosslingual (zero-shot) experiments. The ASR is trained to recognize the International Phonetic Alphabet (IPA) token sequences. We observe significant improvements across all languages in the multilingual setting, and stark degradation in the crosslingual setting, where the model, among other errors, considers Javanese as a tone language. Notably, as little as 10 hours of the target language training data tremendously reduces ASR error rates. Our analysis uncovered that even the phones that are unique to a single language can benefit greatly from adding training data from other languages - an encouraging result for the low-resource speech community.

Related papers

• Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching [65.74653592668743]
  Finetuning self-supervised multilingual representations reduces absolute word error rates by up to 20%. In circumstances with limited training data finetuning self-supervised representations is a better performing and viable solution.
  arXiv  Detail & Related papers  (2023-11-25T17:05:21Z)
• Language-universal phonetic encoder for low-resource speech recognition [28.21805271848413]
  We leverage International Phonetic Alphabet (IPA) based language-universal phonetic model to improve low-resource ASR performances. Our approach and adaptation are effective on extremely low-resource languages, even within domain- and language-mismatched scenarios.
  arXiv  Detail & Related papers  (2023-05-19T10:24:30Z)
• Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages [12.00637655338665]
  We study very low-resource languages and handle 50 African languages, many of which are not covered by any other model. We train sentence encoders, mine bitexts, and validate the bitexts by training NMT systems. For these languages, we train sentence encoders, mine bitexts, and validate the bitexts by training NMT systems.
  arXiv  Detail & Related papers  (2022-05-25T10:53:24Z)
• Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition [71.49308685090324]
  This paper investigates the influence of different factors (i.e., model architecture, phonotactic model, type of speech representation) on phone recognition in an unknown language. We find that unique sounds, similar sounds, and tone languages remain a major challenge for phonetic inventory discovery.
  arXiv  Detail & Related papers  (2022-01-26T22:12:55Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• How Phonotactics Affect Multilingual and Zero-shot ASR Performance [74.70048598292583]
  A Transformer encoder-decoder model has been shown to leverage multilingual data well in IPA transcriptions of languages presented during training. We replace the encoder-decoder with a hybrid ASR system consisting of a separate AM and LM. We show that the gain from modeling crosslingual phonotactics is limited, and imposing a too strong model can hurt the zero-shot transfer.
  arXiv  Detail & Related papers  (2020-10-22T23:07:24Z)
• Improved acoustic word embeddings for zero-resource languages using multilingual transfer [37.78342106714364]
  We train a single supervised embedding model on labelled data from multiple well-resourced languages and apply it to unseen zero-resource languages. We consider three multilingual recurrent neural network (RNN) models: a classifier trained on the joint vocabularies of all training languages; a Siamese RNN trained to discriminate between same and different words from multiple languages; and a correspondence autoencoder (CAE) RNN trained to reconstruct word pairs. All of these models outperform state-of-the-art unsupervised models trained on the zero-resource languages themselves, giving relative improvements of more than 30% in average precision.
  arXiv  Detail & Related papers  (2020-06-02T12:28:34Z)
• Universal Phone Recognition with a Multilingual Allophone System [135.2254086165086]
  We propose a joint model of language-independent phone and language-dependent phoneme distributions. In multilingual ASR experiments over 11 languages, we find that this model improves testing performance by 2% phoneme error rate absolute. Our recognizer achieves phone accuracy improvements of more than 17%, moving a step closer to speech recognition for all languages in the world.
  arXiv  Detail & Related papers  (2020-02-26T21:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.