Adapting to Non-Centered Languages for Zero-shot Multilingual Translation

• URL: http://arxiv.org/abs/2209.04138v1
• Date: Fri, 9 Sep 2022 06:34:12 GMT
• Title: Adapting to Non-Centered Languages for Zero-shot Multilingual Translation
• Authors: Zhi Qu, Taro Watanabe
• Abstract summary: We propose a simple, lightweight yet effective language-specific modeling method by adapting to non-centered languages. Experiments with Transformer on IWSLT17, Europarl, TED talks, and OPUS-100 datasets show that our method can easily fit non-centered data conditions.
• Score: 12.487990897680422
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multilingual neural machine translation can translate unseen language pairs during training, i.e. zero-shot translation. However, the zero-shot translation is always unstable. Although prior works attributed the instability to the domination of central language, e.g. English, we supplement this viewpoint with the strict dependence of non-centered languages. In this work, we propose a simple, lightweight yet effective language-specific modeling method by adapting to non-centered languages and combining the shared information and the language-specific information to counteract the instability of zero-shot translation. Experiments with Transformer on IWSLT17, Europarl, TED talks, and OPUS-100 datasets show that our method not only performs better than strong baselines in centered data conditions but also can easily fit non-centered data conditions. By further investigating the layer attribution, we show that our proposed method can disentangle the coupled representation in the correct direction.

Related papers

• Question Translation Training for Better Multilingual Reasoning [108.10066378240879]
  Large language models show compelling performance on reasoning tasks but they tend to perform much worse in languages other than English. A typical solution is to translate instruction data into all languages of interest, and then train on the resulting multilingual data, which is called translate-training. In this paper we explore the benefits of question alignment, where we train the model to translate reasoning questions into English by finetuning on X-English parallel question data.
  arXiv  Detail & Related papers  (2024-01-15T16:39:10Z)
• Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer [92.80671770992572]
  Cross-lingual transfer is a central task in multilingual NLP. Earlier efforts on this task use parallel corpora, bilingual dictionaries, or other annotated alignment data. We propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-09-19T19:30:56Z)
• Exploring Linguistic Similarity and Zero-Shot Learning for Multilingual Translation of Dravidian Languages [0.34998703934432673]
  We build a single-decoder neural machine translation system for Dravidian-Dravidian multilingual translation. Our model achieves scores within 3 BLEU of large-scale pivot-based models when it is trained on 50% of the language directions.
  arXiv  Detail & Related papers  (2023-08-10T13:38:09Z)
• DEEP: DEnoising Entity Pre-training for Neural Machine Translation [123.6686940355937]
  It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus. We propose DEEP, a DEnoising Entity Pre-training method that leverages large amounts of monolingual data and a knowledge base to improve named entity translation accuracy within sentences.
  arXiv  Detail & Related papers  (2021-11-14T17:28:09Z)
• Improving Multilingual Translation by Representation and Gradient Regularization [82.42760103045083]
  We propose a joint approach to regularize NMT models at both representation-level and gradient-level. Our results demonstrate that our approach is highly effective in both reducing off-target translation occurrences and improving zero-shot translation performance.
  arXiv  Detail & Related papers  (2021-09-10T10:52:21Z)
• Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data [2.225882303328135]
  We propose a novel Translate-and-Fill (TaF) method to produce silver training data for a multilingual semantic parsing task. Experimental results on three multilingual semantic parsing datasets show that data augmentation with TaF reaches accuracies competitive with similar systems.
  arXiv  Detail & Related papers  (2021-09-09T14:51:11Z)
• Improving Zero-Shot Translation by Disentangling Positional Information [24.02434897109097]
  We show that a main factor causing the language-specific representations is the positional correspondence to input tokens. We gain up to 18.5 BLEU points on zero-shot translation while retaining quality on supervised directions.
  arXiv  Detail & Related papers  (2020-12-30T12:20:41Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.