Cross-lingual Supervision Improves Unsupervised Neural Machine Translation

• URL: http://arxiv.org/abs/2004.03137v3
• Date: Thu, 1 Apr 2021 03:52:50 GMT
• Title: Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
• Authors: Mingxuan Wang, Hongxiao Bai, Hai Zhao, Lei Li
• Abstract summary: We introduce a multilingual unsupervised NMT framework to leverage weakly supervised signals from high-resource language pairs to zero-resource translation directions. Method significantly improves the translation quality by more than 3 BLEU score on six benchmark unsupervised translation directions.
• Score: 97.84871088440102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural machine translation~(NMT) is ineffective for zero-resource languages. Recent works exploring the possibility of unsupervised neural machine translation (UNMT) with only monolingual data can achieve promising results. However, there are still big gaps between UNMT and NMT with parallel supervision. In this work, we introduce a multilingual unsupervised NMT (\method) framework to leverage weakly supervised signals from high-resource language pairs to zero-resource translation directions. More specifically, for unsupervised language pairs \texttt{En-De}, we can make full use of the information from parallel dataset \texttt{En-Fr} to jointly train the unsupervised translation directions all in one model. \method is based on multilingual models which require no changes to the standard unsupervised NMT. Empirical results demonstrate that \method significantly improves the translation quality by more than 3 BLEU score on six benchmark unsupervised translation directions.

Related papers

• Towards Zero-Shot Multimodal Machine Translation [64.9141931372384]
  We propose a method to bypass the need for fully supervised data to train multimodal machine translation systems. Our method, called ZeroMMT, consists in adapting a strong text-only machine translation (MT) model by training it on a mixture of two objectives. To prove that our method generalizes to languages with no fully supervised training data available, we extend the CoMMuTE evaluation dataset to three new languages: Arabic, Russian and Chinese.
  arXiv  Detail & Related papers  (2024-07-18T15:20:31Z)
• An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contribution [5.691028372215281]
  Unsupervised Neural Machine Translation (UNMT) focuses on improving NMT results under the assumption there is no human translated parallel data. We focus on three very diverse languages, French, Gujarati, and Kazakh, and train bilingual NMT models, to and from English, with various levels of supervision.
  arXiv  Detail & Related papers  (2023-12-19T20:35:08Z)
• Boosting Unsupervised Machine Translation with Pseudo-Parallel Data [2.900810893770134]
  We propose a training strategy that relies on pseudo-parallel sentence pairs mined from monolingual corpora and synthetic sentence pairs back-translated from monolingual corpora. We reach an improvement of up to 14.5 BLEU points (English to Ukrainian) over a baseline trained on back-translated data only.
  arXiv  Detail & Related papers  (2023-10-22T10:57:12Z)
• Improving Multilingual Translation by Representation and Gradient Regularization [82.42760103045083]
  We propose a joint approach to regularize NMT models at both representation-level and gradient-level. Our results demonstrate that our approach is highly effective in both reducing off-target translation occurrences and improving zero-shot translation performance.
  arXiv  Detail & Related papers  (2021-09-10T10:52:21Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation [54.52971020087777]
  Using monolingual data significantly boosts the translation quality of low-resource languages in multilingual models. Self-supervision improves zero-shot translation quality in multilingual models. We get up to 33 BLEU on ro-en translation without any parallel data or back-translation.
  arXiv  Detail & Related papers  (2020-05-11T00:20:33Z)
• When and Why is Unsupervised Neural Machine Translation Useless? [43.68079166777282]
  In ten translation tasks with various data settings, we analyze the conditions under which the unsupervised methods fail to produce reasonable translations. Our analyses pinpoint the limits of the current unsupervised NMT and also suggest immediate research directions.
  arXiv  Detail & Related papers  (2020-04-22T14:00:55Z)
• Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios [61.88012735215636]
  Unsupervised neural machine translation (UNMT) that relies solely on massive monolingual corpora has achieved remarkable results in several translation tasks. In real-world scenarios, massive monolingual corpora do not exist for some extremely low-resource languages such as Estonian. We propose UNMT self-training mechanisms to train a robust UNMT system and improve its performance.
  arXiv  Detail & Related papers  (2020-04-09T12:07:17Z)
• A Multilingual View of Unsupervised Machine Translation [22.32130421893608]
  We present a probabilistic framework for multilingual neural machine translation that encompasses supervised and unsupervised setups. We show that our approach results in higher BLEU scores over state-of-the-art unsupervised models on the WMT'14 English-French, WMT'16 English-German, and WMT'16 English-Romanian datasets.
  arXiv  Detail & Related papers  (2020-02-07T18:50:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.