Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer

• URL: http://arxiv.org/abs/2109.13097v1
• Date: Mon, 27 Sep 2021 14:49:35 GMT
• Title: Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer
• Authors: Evgeniia Tokarchuk, Jan Rosendahl, Weiyue Wang, Pavel Petrushkov, Tomer Lancewicki, Shahram Khadivi, Hermann Ney
• Abstract summary: Pivot-based neural machine translation (NMT) is commonly used in low-resource setups. We present an end-to-end pivot-based integrated model, enabling training on source-target data.
• Score: 49.897891031932545
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pivot-based neural machine translation (NMT) is commonly used in low-resource setups, especially for translation between non-English language pairs. It benefits from using high resource source-pivot and pivot-target language pairs and an individual system is trained for both sub-tasks. However, these models have no connection during training, and the source-pivot model is not optimized to produce the best translation for the source-target task. In this work, we propose to train a pivot-based NMT system with the reinforcement learning (RL) approach, which has been investigated for various text generation tasks, including machine translation (MT). We utilize a non-autoregressive transformer and present an end-to-end pivot-based integrated model, enabling training on source-target data.

Related papers

• Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation [19.28500206536013]
  Federated learning (FL) is a promising approach for solving multilingual tasks. We propose a meta-learning-based adaptive parameter selection methodology, MetaSend, that improves the communication efficiency of model transmissions. We demonstrate that MetaSend obtains substantial improvements over baselines in translation quality in the presence of a limited communication budget.
  arXiv  Detail & Related papers  (2024-01-15T04:04:26Z)
• End-to-end Training and Decoding for Pivot-based Cascaded Translation Model [40.41344631506705]
  We propose an end-to-end training method for the cascaded translation model. We mitigate the inconsistency between tokens and probability distributions while using beam search in pivot decoding.
  arXiv  Detail & Related papers  (2023-05-03T16:48:43Z)
• Active Learning for Neural Machine Translation [0.0]
  We incorporated a technique known Active Learning with the NMT toolkit Joey NMT to reach sufficient accuracy and robust predictions of low-resource language translation. This work uses transformer-based NMT systems; baseline model (BM), fully trained model (FTM), active learning least confidence based model (ALLCM) and active learning margin sampling based model (ALMSM) when translating English to Hindi.
  arXiv  Detail & Related papers  (2022-12-30T17:04:01Z)
• Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation [50.54059385277964]
  We present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CsaNMT) CsaNMT augments each training instance with an adjacency region that could cover adequate variants of literal expression under the same meaning.
  arXiv  Detail & Related papers  (2022-04-14T08:16:28Z)
• Distributionally Robust Multilingual Machine Translation [94.51866646879337]
  We propose a new learning objective for Multilingual neural machine translation (MNMT) based on distributionally robust optimization. We show how to practically optimize this objective for large translation corpora using an iterated best response scheme. Our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.
  arXiv  Detail & Related papers  (2021-09-09T03:48:35Z)
• Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT [64.1841519527504]
  neural machine translation uses a single neural network to model the entire translation process. Despite neural machine translation being de-facto standard, it is still not clear how NMT models acquire different competences over the course of training.
  arXiv  Detail & Related papers  (2021-09-03T09:38:50Z)
• Simultaneous Multi-Pivot Neural Machine Translation [12.796775798210133]
  In a simultaneous pivot NMT setting, using two pivot languages can lead to an improvement of up to 5.8 BLEU. Our experiments in a low-resource setting using the N-way parallel UN corpus for Arabic to English NMT via French and Spanish as pivots reveals that in a simultaneous pivot NMT setting, using two pivot languages can lead to an improvement of up to 5.8 BLEU.
  arXiv  Detail & Related papers  (2021-04-15T12:19:52Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation [59.191079800436114]
  Document-level machine translation incorporates inter-sentential dependencies into the translation of a source sentence. We propose a new framework to model cross-sentence dependencies by training neural machine translation (NMT) to predict both the target translation and surrounding sentences of a source sentence.
  arXiv  Detail & Related papers  (2020-03-30T03:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.