Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training

• URL: http://arxiv.org/abs/2002.12549v2
• Date: Thu, 3 Dec 2020 03:19:36 GMT
• Title: Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training
• Authors: Haipeng Sun, Rui Wang, Kehai Chen, Xugang Lu, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao
• Abstract summary: Unsupervised neural machine translation (UNMT) has attracted great interest in the machine translation community. The main advantage of the UNMT lies in its easy collection of required large training text sentences. In this paper, we first time explicitly take the noisy data into consideration to improve the robustness of the UNMT based systems.
• Score: 66.39561682517741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised neural machine translation (UNMT) has recently attracted great interest in the machine translation community. The main advantage of the UNMT lies in its easy collection of required large training text sentences while with only a slightly worse performance than supervised neural machine translation which requires expensive annotated translation pairs on some translation tasks. In most studies, the UMNT is trained with clean data without considering its robustness to the noisy data. However, in real-world scenarios, there usually exists noise in the collected input sentences which degrades the performance of the translation system since the UNMT is sensitive to the small perturbations of the input sentences. In this paper, we first time explicitly take the noisy data into consideration to improve the robustness of the UNMT based systems. First of all, we clearly defined two types of noises in training sentences, i.e., word noise and word order noise, and empirically investigate its effect in the UNMT, then we propose adversarial training methods with denoising process in the UNMT. Experimental results on several language pairs show that our proposed methods substantially improved the robustness of the conventional UNMT systems in noisy scenarios.

Related papers

• How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise on Machine Translation [10.739338438716965]
  We study the impact of real-world hard-to-detect misalignment noise on machine translation. By observing the increasing reliability of the model's self-knowledge for distinguishing misaligned and clean data at the token-level, we propose a self-correction approach. Our method proves effective for real-world noisy web-mined datasets across eight translation tasks.
  arXiv  Detail & Related papers  (2024-07-02T12:15:15Z)
• Ask Language Model to Clean Your Noisy Translation Data [7.246698449812031]
  We focus on cleaning the noise from the target sentences in MTNT, making it more suitable as a benchmark for noise evaluation. We show that large language models (LLMs) can effectively rephrase slang, jargon, and profanities. Experiments on C-MTNT showcased its effectiveness in evaluating the robustness of NMT models.
  arXiv  Detail & Related papers  (2023-10-20T13:05:32Z)
• Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation [70.33052952571884]
  We propose to build a cascaded speech translation system without leveraging any kind of paired data. We use fully unpaired data to train our unsupervised systems and evaluate our results on CoVoST 2 and CVSS.
  arXiv  Detail & Related papers  (2023-05-12T13:07:51Z)
• DEEP: DEnoising Entity Pre-training for Neural Machine Translation [123.6686940355937]
  It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus. We propose DEEP, a DEnoising Entity Pre-training method that leverages large amounts of monolingual data and a knowledge base to improve named entity translation accuracy within sentences.
  arXiv  Detail & Related papers  (2021-11-14T17:28:09Z)
• Phrase-level Active Learning for Neural Machine Translation [107.28450614074002]
  We propose an active learning setting where we can spend a given budget on translating in-domain data. We select both full sentences and individual phrases from unlabelled data in the new domain for routing to human translators. In a German-English translation task, our active learning approach achieves consistent improvements over uncertainty-based sentence selection methods.
  arXiv  Detail & Related papers  (2021-06-21T19:20:42Z)
• Addressing the Vulnerability of NMT in Input Perturbations [10.103375853643547]
  We improve robustness of NMT models by reducing the effect of noisy words through a Context-Enhanced Reconstruction approach. CER trains the model to resist noise in two steps: (1) step that breaks the naturalness of input sequence with made-up words; (2) reconstruction step that defends the noise propagation by generating better and more robust contextual representation.
  arXiv  Detail & Related papers  (2021-04-20T07:52:58Z)
• Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios [61.88012735215636]
  Unsupervised neural machine translation (UNMT) that relies solely on massive monolingual corpora has achieved remarkable results in several translation tasks. In real-world scenarios, massive monolingual corpora do not exist for some extremely low-resource languages such as Estonian. We propose UNMT self-training mechanisms to train a robust UNMT system and improve its performance.
  arXiv  Detail & Related papers  (2020-04-09T12:07:17Z)
• Multilingual Denoising Pre-training for Neural Machine Translation [132.66750663226287]
  mBART is a sequence-to-sequence denoising auto-encoder pre-trained on large-scale monolingual corpora. mBART is one of the first methods for pre-training a complete sequence-to-sequence model.
  arXiv  Detail & Related papers  (2020-01-22T18:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.