Triangular Transfer: Freezing the Pivot for Triangular Machine Translation

• URL: http://arxiv.org/abs/2203.09027v1
• Date: Thu, 17 Mar 2022 02:00:40 GMT
• Title: Triangular Transfer: Freezing the Pivot for Triangular Machine Translation
• Authors: Meng Zhang, Liangyou Li, Qun Liu
• Abstract summary: Triangular machine translation is a case where the language pair of interest has limited parallel data. Key to triangular machine translation is the successful exploitation of such auxiliary data. We propose a transfer-learning-based approach that utilizes all types of auxiliary data.
• Score: 30.655004159965923
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Triangular machine translation is a special case of low-resource machine translation where the language pair of interest has limited parallel data, but both languages have abundant parallel data with a pivot language. Naturally, the key to triangular machine translation is the successful exploitation of such auxiliary data. In this work, we propose a transfer-learning-based approach that utilizes all types of auxiliary data. As we train auxiliary source-pivot and pivot-target translation models, we initialize some parameters of the pivot side with a pre-trained language model and freeze them to encourage both translation models to work in the same pivot language space, so that they can be smoothly transferred to the source-target translation model. Experiments show that our approach can outperform previous ones.

Related papers

• End-to-end Training and Decoding for Pivot-based Cascaded Translation Model [40.41344631506705]
  We propose an end-to-end training method for the cascaded translation model. We mitigate the inconsistency between tokens and probability distributions while using beam search in pivot decoding.
  arXiv  Detail & Related papers  (2023-05-03T16:48:43Z)
• On the Role of Parallel Data in Cross-lingual Transfer Learning [30.737717433111776]
  We examine the usage of unsupervised machine translation to generate synthetic parallel data. We find that even model generated parallel data can be useful for downstream tasks. Our findings suggest that existing multilingual models do not exploit the full potential of monolingual data.
  arXiv  Detail & Related papers  (2022-12-20T11:23:04Z)
• Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation [23.401781865904386]
  We propose a two-stage approach for training a single NMT model to translate unseen languages both to and from English. For the first stage, we initialize an encoder-decoder model to pretrained XLM-R and RoBERTa weights, then perform multilingual fine-tuning on parallel data in 40 languages to English. For the second stage, we leverage this generalization ability to generate synthetic parallel data from monolingual datasets, then bidirectionally train with successive rounds of back-translation.
  arXiv  Detail & Related papers  (2022-09-06T21:20:41Z)
• Principled Paraphrase Generation with Parallel Corpora [52.78059089341062]
  We formalize the implicit similarity function induced by round-trip Machine Translation. We show that it is susceptible to non-paraphrase pairs sharing a single ambiguous translation. We design an alternative similarity metric that mitigates this issue.
  arXiv  Detail & Related papers  (2022-05-24T17:22:42Z)
• Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer [49.897891031932545]
  Pivot-based neural machine translation (NMT) is commonly used in low-resource setups. We present an end-to-end pivot-based integrated model, enabling training on source-target data.
  arXiv  Detail & Related papers  (2021-09-27T14:49:35Z)
• Improving Neural Machine Translation by Bidirectional Training [85.64797317290349]
  We present a simple and effective pretraining strategy -- bidirectional training (BiT) for neural machine translation. Specifically, we bidirectionally update the model parameters at the early stage and then tune the model normally. Experimental results show that BiT pushes the SOTA neural machine translation performance across 15 translation tasks on 8 language pairs significantly higher.
  arXiv  Detail & Related papers  (2021-09-16T07:58:33Z)
• Simultaneous Multi-Pivot Neural Machine Translation [12.796775798210133]
  In a simultaneous pivot NMT setting, using two pivot languages can lead to an improvement of up to 5.8 BLEU. Our experiments in a low-resource setting using the N-way parallel UN corpus for Arabic to English NMT via French and Spanish as pivots reveals that in a simultaneous pivot NMT setting, using two pivot languages can lead to an improvement of up to 5.8 BLEU.
  arXiv  Detail & Related papers  (2021-04-15T12:19:52Z)
• XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders [89.0059978016914]
  We present XLM-T, which initializes the model with an off-the-shelf pretrained cross-lingual Transformer and fine-tunes it with multilingual parallel data. This simple method achieves significant improvements on a WMT dataset with 10 language pairs and the OPUS-100 corpus with 94 pairs.
  arXiv  Detail & Related papers  (2020-12-31T11:16:51Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)
• Exemplar-Controllable Paraphrasing and Translation using Bitext [57.92051459102902]
  We adapt models from prior work to be able to learn solely from bilingual text (bitext) Our single proposed model can perform four tasks: controlled paraphrase generation in both languages and controlled machine translation in both language directions.
  arXiv  Detail & Related papers  (2020-10-12T17:02:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.