On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss

• URL: http://arxiv.org/abs/2305.17182v2
• Date: Sun, 4 Jun 2023 09:41:35 GMT
• Title: On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss
• Authors: Yihong Liu, Alexandra Chronopoulou, Hinrich Sch\"utze, Alexander Fraser
• Abstract summary: unsupervised neural machine translation (UNMT) has achieved success in many language pairs. The copying problem, i.e., directly copying some parts of the input sentence as the translation, is common among distant language pairs. We propose a simple but effective training schedule that incorporates a language discriminator loss.
• Score: 120.19360680963152
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although unsupervised neural machine translation (UNMT) has achieved success in many language pairs, the copying problem, i.e., directly copying some parts of the input sentence as the translation, is common among distant language pairs, especially when low-resource languages are involved. We find this issue is closely related to an unexpected copying behavior during online back-translation (BT). In this work, we propose a simple but effective training schedule that incorporates a language discriminator loss. The loss imposes constraints on the intermediate translation so that the translation is in the desired language. By conducting extensive experiments on different language pairs, including similar and distant, high and low-resource languages, we find that our method alleviates the copying problem, thus improving the translation performance on low-resource languages.

Related papers

• A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representations [0.4499833362998489]
  This study focuses on the case of English-Marathi language pairs, where existing datasets are notably noisy. To mitigate the impact of data quality issues, we propose a data filtering approach based on cross-lingual sentence representations. Results demonstrate a significant improvement in translation quality over the baseline post-filtering with IndicSBERT.
  arXiv  Detail & Related papers  (2024-09-04T13:49:45Z)
• Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment [50.27950279695363]
  The transfer performance is often hindered when a low-resource target language is written in a different script than the high-resource source language. Inspired by recent work that uses transliteration to address this problem, our paper proposes a transliteration-based post-pretraining alignment (PPA) method.
  arXiv  Detail & Related papers  (2024-06-28T08:59:24Z)
• Extending Multilingual Machine Translation through Imitation Learning [60.15671816513614]
  Imit-MNMT treats the task as an imitation learning process, which mimicks the behavior of an expert. We show that our approach significantly improves the translation performance between the new and the original languages. We also demonstrate that our approach is capable of solving copy and off-target problems.
  arXiv  Detail & Related papers  (2023-11-14T21:04:03Z)
• The Best of Both Worlds: Combining Human and Machine Translations for Multilingual Semantic Parsing with Active Learning [50.320178219081484]
  We propose an active learning approach that exploits the strengths of both human and machine translations. An ideal utterance selection can significantly reduce the error and bias in the translated data.
  arXiv  Detail & Related papers  (2023-05-22T05:57:47Z)
• On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation [104.85258654917297]
  We find that failing in encoding discriminative target language signal will lead to off-target and a closer lexical distance. We propose Language Aware Vocabulary Sharing (LAVS) to construct the multilingual vocabulary. We conduct experiments on a multilingual machine translation benchmark in 11 languages.
  arXiv  Detail & Related papers  (2023-05-18T12:43:31Z)
• Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation [61.88012735215636]
  Unsupervised neural machine translation (UNMT) has recently achieved remarkable results for several language pairs. UNMT can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time. In this paper, we empirically introduce a simple method to translate between thirteen languages using a single encoder and a single decoder.
  arXiv  Detail & Related papers  (2020-04-21T17:26:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.