Exploiting Adapters for Cross-lingual Low-resource Speech Recognition

• URL: http://arxiv.org/abs/2105.11905v1
• Date: Tue, 18 May 2021 08:30:37 GMT
• Title: Exploiting Adapters for Cross-lingual Low-resource Speech Recognition
• Authors: Wenxin Hou, Han Zhu, Yidong Wang, Jindong Wang, Tao Qin, Renjun Xu, Takahiro Shinozaki
• Abstract summary: Cross-lingual speech adaptation aims to solve the problem of leveraging multiple rich-resource languages to build models for a low-resource target language. We propose adapters to investigate the performance of multiple adapters for parameter-efficient cross-lingual speech adaptation.
• Score: 52.40623653290499
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-lingual speech adaptation aims to solve the problem of leveraging multiple rich-resource languages to build models for a low-resource target language. Since the low-resource language has limited training data, speech recognition models can easily overfit. In this paper, we propose to use adapters to investigate the performance of multiple adapters for parameter-efficient cross-lingual speech adaptation. Based on our previous MetaAdapter that implicitly leverages adapters, we propose a novel algorithms called SimAdapter for explicitly learning knowledge from adapters. Our algorithm leverages adapters which can be easily integrated into the Transformer structure.MetaAdapter leverages meta-learning to transfer the general knowledge from training data to the test language. SimAdapter aims to learn the similarities between the source and target languages during fine-tuning using the adapters. We conduct extensive experiments on five-low-resource languages in Common Voice dataset. Results demonstrate that our MetaAdapter and SimAdapter methods can reduce WER by 2.98% and 2.55% with only 2.5% and 15.5% of trainable parameters compared to the strong full-model fine-tuning baseline. Moreover, we also show that these two novel algorithms can be integrated for better performance with up to 3.55% relative WER reduction.

Related papers

• ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks [10.852047082856487]
  We introduce ELP-adapter tuning, a novel method for parameter-efficient fine-tuning using three types of adapters. E-adapters are integrated into transformer-based encoder layers and help to learn fine-grained speech representations that are effective for speech recognition. L-adapters create paths from each encoder layer to the downstream head and help to extract non-linguistic features from lower encoder layers. The P-adapter appends pseudo features to CNN features to further improve effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-07-28T05:26:03Z)
• The Impact of Language Adapters in Cross-Lingual Transfer for NLU [0.8702432681310401]
  We study the effect of including a target-language adapter in detailed ablation studies with two multilingual models and three multilingual datasets. Our results show that the effect of target-language adapters is highly inconsistent across tasks, languages and models. Removing the language adapter after training has only a weak negative effect, indicating that the language adapters do not have a strong impact on the predictions.
  arXiv  Detail & Related papers  (2024-01-31T20:07:43Z)
• MerA: Merging Pretrained Adapters For Few-Shot Learning [71.44422347502409]
  We propose textbftextttMerging Pretrained Adapters (MerA) that efficiently incorporates pretrained adapters to a single model through model fusion. Experiments on two PLMs demonstrate that MerA substantial improvements compared to both single adapters and AdapterFusion.
  arXiv  Detail & Related papers  (2023-08-30T12:10:17Z)
• Language-Universal Adapter Learning with Knowledge Distillation for End-to-End Multilingual Speech Recognition [28.416831396722106]
  We propose a language-universal adapter learning framework based on a pre-trained model for end-to-end multilingual automatic speech recognition. An online knowledge distillation is then used to enable the language-universal adapters to learn both language-specific and universal features. Compared to the conventional multilingual model, a 3.3% absolute error rate reduction is achieved.
  arXiv  Detail & Related papers  (2023-02-28T14:43:49Z)
• Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation [129.99918589405675]
  Large multilingual models trained with self-supervision achieve state-of-the-art results in a wide range of natural language processing tasks. Multilingual fine-tuning improves performance on low-resource languages but requires modifying the entire model and can be prohibitively expensive. We propose training language-family adapters on top of mBART-50 to facilitate cross-lingual transfer.
  arXiv  Detail & Related papers  (2022-09-30T05:02:42Z)
• VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks [71.40656211497162]
  Recently, fine-tuning language models pre-trained on large text corpora have provided huge improvements on vision-and-language (V&L) tasks. We introduce adapter-based parameter-efficient transfer learning techniques to V&L models such as VL-BART and VL-T5. Our results demonstrate that training the adapter with the weight-sharing technique can match the performance of fine-tuning the entire model.
  arXiv  Detail & Related papers  (2021-12-13T17:35:26Z)
• Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters [66.7986513246294]
  We study the compositionality of language and domain adapters in the context of Machine Translation. We find that in the partial resource scenario a naive combination of domain-specific and language-specific adapters often results in catastrophic forgetting' of the missing languages.
  arXiv  Detail & Related papers  (2021-10-18T18:55:23Z)
• Efficient Test Time Adapter Ensembling for Low-resource Language Varieties [115.12997212870962]
  Specialized language and task adapters have been proposed to facilitate cross-lingual transfer of multilingual pretrained models. An intuitive solution is to use a related language adapter for the new language variety, but we observe that this solution can lead to sub-optimal performance. In this paper, we aim to improve the robustness of language adapters to uncovered languages without training new adapters.
  arXiv  Detail & Related papers  (2021-09-10T13:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.