Machine Translation Advancements of Low-Resource Indian Languages by Transfer Learning

• URL: http://arxiv.org/abs/2409.15879v1
• Date: Tue, 24 Sep 2024 08:53:19 GMT
• Title: Machine Translation Advancements of Low-Resource Indian Languages by Transfer Learning
• Authors: Bin Wei, Jiawei Zhen, Zongyao Li, Zhanglin Wu, Daimeng Wei, Jiaxin Guo, Zhiqiang Rao, Shaojun Li, Yuanchang Luo, Hengchao Shang, Jinlong Yang, Yuhao Xie, Hao Yang,
• Abstract summary: We employ two distinct knowledge transfer strategies to develop a reliable machine translation system for low-resource Indian languages. For Assamese(as) and Manipuri(mn), we fine-tuned the existing IndicTrans2 open-source model to enable bidirectional translation between English and these languages. For Khasi (kh) and Mizo (mz), We trained a multilingual model as a baseline using bilingual data from these four language pairs, along with an additional about 8kw English-Bengali bilingual data.
• Score: 9.373815852241648
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper introduces the submission by Huawei Translation Center (HW-TSC) to the WMT24 Indian Languages Machine Translation (MT) Shared Task. To develop a reliable machine translation system for low-resource Indian languages, we employed two distinct knowledge transfer strategies, taking into account the characteristics of the language scripts and the support available from existing open-source models for Indian languages. For Assamese(as) and Manipuri(mn), we fine-tuned the existing IndicTrans2 open-source model to enable bidirectional translation between English and these languages. For Khasi (kh) and Mizo (mz), We trained a multilingual model as a baseline using bilingual data from these four language pairs, along with an additional about 8kw English-Bengali bilingual data, all of which share certain linguistic features. This was followed by fine-tuning to achieve bidirectional translation between English and Khasi, as well as English and Mizo. Our transfer learning experiments produced impressive results: 23.5 BLEU for en-as, 31.8 BLEU for en-mn, 36.2 BLEU for as-en, and 47.9 BLEU for mn-en on their respective test sets. Similarly, the multilingual model transfer learning experiments yielded impressive outcomes, achieving 19.7 BLEU for en-kh, 32.8 BLEU for en-mz, 16.1 BLEU for kh-en, and 33.9 BLEU for mz-en on their respective test sets. These results not only highlight the effectiveness of transfer learning techniques for low-resource languages but also contribute to advancing machine translation capabilities for low-resource Indian languages.

Related papers

• Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Language [1.1702440973773898]
  This study explores the use of large language models for translating English into Mambai, a low-resource Austronesian language spoken in Timor-Leste. Our methodology involves the strategic selection of parallel sentences and dictionary entries for prompting. We find that including dictionary entries in prompts and a mix of sentences retrieved through-IDF and semantic embeddings significantly improves translation quality.
  arXiv  Detail & Related papers  (2024-04-07T05:04:38Z)
• Machine Translation for Ge'ez Language [0.0]
  Machine translation for low-resource languages such as Ge'ez faces challenges such as out-of-vocabulary words, domain mismatches, and lack of labeled training data. We develop a multilingual neural machine translation (MNMT) model based on languages relatedness. We also experiment with using GPT-3.5, a state-of-the-art LLM, for few-shot translation with fuzzy matches.
  arXiv  Detail & Related papers  (2023-11-24T14:55:23Z)
• Baichuan 2: Open Large-scale Language Models [51.56361715162972]
  We present Baichuan 2, a series of large-scale multilingual language models containing 7 billion and 13 billion parameters, trained from scratch, on 2.6 trillion tokens. Baichuan 2 matches or outperforms other open-source models of similar size on public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval.
  arXiv  Detail & Related papers  (2023-09-19T04:13:22Z)
• Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts [75.33019401706188]
  Large language models (LLMs) are known to effectively perform tasks by simply observing few exemplars. We propose to assemble synthetic exemplars from a diverse set of high-resource languages to prompt the LLMs to translate from any language into English. Our unsupervised prompting method performs on par with supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages.
  arXiv  Detail & Related papers  (2023-06-20T08:27:47Z)
• Enhancing Translation for Indigenous Languages: Experiments with Multilingual Models [57.10972566048735]
  We present the system descriptions for three methods. We used two multilingual models, namely M2M-100 and mBART50, and one bilingual (one-to-one) -- Helsinki NLP Spanish-English translation model. We experimented with 11 languages from America and report the setups we used as well as the results we achieved.
  arXiv  Detail & Related papers  (2023-05-27T08:10:40Z)
• Parallel Corpus for Indigenous Language Translation: Spanish-Mazatec and Spanish-Mixtec [51.35013619649463]
  We present a parallel Spanish-Mazatec and Spanish-Mixtec corpus for machine translation (MT) tasks. We evaluated the usability of the collected corpus using three different approaches: transformer, transfer learning, and fine-tuning pre-trained multilingual MT models. The findings show that the dataset size (9,799 sentences in Mazatec and 13,235 sentences in Mixtec) affects translation performance and that indigenous languages work better when used as target languages.
  arXiv  Detail & Related papers  (2023-05-27T08:03:44Z)
• Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs [0.0]
  We focus on the three relatively low resource language pairs Bengali to and from Hindi, English to and from Hausa, and Xhosa to and from Zulu. We train a multilingual model using a multitask objective employing both parallel and monolingual data. We see around 70% relative gain in BLEU point for English to and from Hausa, and around 25% relative improvements for both Bengali to and from Hindi, and Xhosa to and from Zulu.
  arXiv  Detail & Related papers  (2021-11-26T00:54:37Z)
• Cross-Lingual Training with Dense Retrieval for Document Retrieval [56.319511218754414]
  We explore different transfer techniques for document ranking from English annotations to multiple non-English languages. Experiments on the test collections in six languages (Chinese, Arabic, French, Hindi, Bengali, Spanish) from diverse language families. We find that weakly-supervised target language transfer yields competitive performances against the generation-based target language transfer.
  arXiv  Detail & Related papers  (2021-09-03T17:15:38Z)
• Bilingual Dictionary-based Language Model Pretraining for Neural Machine Translation [0.0]
  We incorporate the translation information from dictionaries into the pretraining process and propose a novel Bilingual Dictionary-based Language Model (BDLM) We evaluate our BDLM in Chinese, English, and Romanian.
  arXiv  Detail & Related papers  (2021-03-12T02:01:22Z)
• Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings [72.69253034282035]
  We exploit a language independent multilingual sentence representation to easily generalize to a new language. Blindly decoding from Portuguese using a basesystem containing several Romance languages we achieve scores of 36.4 BLEU for Portuguese-English and 12.8 BLEU for Russian-English. We explore a more practical adaptation approach through non-iterative backtranslation, exploiting our model's ability to produce high quality translations.
  arXiv  Detail & Related papers  (2021-03-11T14:22:08Z)
• Neural Machine Translation for Low-Resourced Indian Languages [4.726777092009554]
  Machine translation is an effective approach to convert text to a different language without any human involvement. In this paper, we have applied NMT on two of the most morphological rich Indian languages, i.e. English-Tamil and English-Malayalam. We proposed a novel NMT model using Multihead self-attention along with pre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an efficient translation system.
  arXiv  Detail & Related papers  (2020-04-19T17:29:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.