Learning Policies for Multilingual Training of Neural Machine Translation Systems

• URL: http://arxiv.org/abs/2103.06964v1
• Date: Thu, 11 Mar 2021 21:38:04 GMT
• Title: Learning Policies for Multilingual Training of Neural Machine Translation Systems
• Authors: Gaurav Kumar, Philipp Koehn, Sanjeev Khudanpur
• Abstract summary: Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs. We propose two simple search based curricula, which help improve translation performance in conjunction with existing techniques such as fine-tuning.
• Score: 36.292020779233056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper, we propose two simple search based curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally, we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system with the aid of contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.

Related papers

• TasTe: Teaching Large Language Models to Translate through Self-Reflection [82.83958470745381]
  Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. We propose the TasTe framework, which stands for translating through self-reflection. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-12T17:21:21Z)
• Relevance-guided Neural Machine Translation [5.691028372215281]
  We propose an explainability-based training approach for Neural Machine Translation (NMT) Our results show our method can be promising, particularly when training in low-resource conditions.
  arXiv  Detail & Related papers  (2023-11-30T21:52:02Z)
• Boosting Unsupervised Machine Translation with Pseudo-Parallel Data [2.900810893770134]
  We propose a training strategy that relies on pseudo-parallel sentence pairs mined from monolingual corpora and synthetic sentence pairs back-translated from monolingual corpora. We reach an improvement of up to 14.5 BLEU points (English to Ukrainian) over a baseline trained on back-translated data only.
  arXiv  Detail & Related papers  (2023-10-22T10:57:12Z)
• Translation-Enhanced Multilingual Text-to-Image Generation [61.41730893884428]
  Research on text-to-image generation (TTI) still predominantly focuses on the English language. In this work, we thus investigate multilingual TTI and the current potential of neural machine translation (NMT) to bootstrap mTTI systems. We propose Ensemble Adapter (EnsAd), a novel parameter-efficient approach that learns to weigh and consolidate the multilingual text knowledge within the mTTI framework.
  arXiv  Detail & Related papers  (2023-05-30T17:03:52Z)
• Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations [75.73028056136778]
  We show how to practically build MNMT systems that serve arbitrary X-Y translation directions. We also examine our proposed approach in an extremely large-scale data setting to accommodate practical deployment scenarios.
  arXiv  Detail & Related papers  (2022-06-30T02:18:15Z)
• Extremely low-resource machine translation for closely related languages [0.0]
  This work focuses on closely related languages from the Uralic language family: from Estonian and Finnish. We find that multilingual learning and synthetic corpora increase the translation quality in every language pair. We show that transfer learning and fine-tuning are very effective for doing low-resource machine translation and achieve the best results.
  arXiv  Detail & Related papers  (2021-05-27T11:27:06Z)
• Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation [53.22775597051498]
  We present a continual pre-training framework on mBART to effectively adapt it to unseen languages. Results show that our method can consistently improve the fine-tuning performance upon the mBART baseline. Our approach also boosts the performance on translation pairs where both languages are seen in the original mBART's pre-training.
  arXiv  Detail & Related papers  (2021-05-09T14:49:07Z)
• Improving Multilingual Neural Machine Translation For Low-Resource Languages: French-, English- Vietnamese [4.103253352106816]
  This paper proposes two simple strategies to address the rare word issue in multilingual MT systems for two low-resource language pairs: French-Vietnamese and English-Vietnamese. We have shown significant improvements of up to +1.62 and +2.54 BLEU points over the bilingual baseline systems for both language pairs.
  arXiv  Detail & Related papers  (2020-12-16T04:43:43Z)
• Multi-task Learning for Multilingual Neural Machine Translation [32.81785430242313]
  We propose a multi-task learning framework that jointly trains the model with the translation task on bitext data and two denoising tasks on the monolingual data. We show that the proposed approach can effectively improve the translation quality for both high-resource and low-resource languages.
  arXiv  Detail & Related papers  (2020-10-06T06:54:12Z)
• Self-Supervised Representations Improve End-to-End Speech Translation [57.641761472372814]
  We show that self-supervised pre-trained features can consistently improve the translation performance. Cross-lingual transfer allows to extend to a variety of languages without or with little tuning.
  arXiv  Detail & Related papers  (2020-06-22T10:28:38Z)
• Pre-training via Leveraging Assisting Languages and Data Selection for Neural Machine Translation [49.51278300110449]
  We propose to exploit monolingual corpora of other languages to complement the scarcity of monolingual corpora for the languages of interest. A case study of low-resource Japanese-English neural machine translation (NMT) reveals that leveraging large Chinese and French monolingual corpora can help overcome the shortage of Japanese and English monolingual corpora.
  arXiv  Detail & Related papers  (2020-01-23T02:47:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.