A Template-based Method for Constrained Neural Machine Translation

• URL: http://arxiv.org/abs/2205.11255v1
• Date: Mon, 23 May 2022 12:24:34 GMT
• Title: A Template-based Method for Constrained Neural Machine Translation
• Authors: Shuo Wang, Peng Li, Zhixing Tan, Zhaopeng Tu, Maosong Sun, Yang Liu
• Abstract summary: We propose a template-based method that can yield results with high translation quality and match accuracy while keeping the decoding speed. The generation and derivation of the template can be learned through one sequence-to-sequence training framework. Experimental results show that the proposed template-based methods can outperform several representative baselines in lexically and structurally constrained translation tasks.
• Score: 100.02590022551718
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine translation systems are expected to cope with various types of constraints in many practical scenarios. While neural machine translation (NMT) has achieved strong performance in unconstrained cases, it is non-trivial to impose pre-specified constraints into the translation process of NMT models. Although many approaches have been proposed to address this issue, most existing methods can not satisfy the following three desiderata at the same time: (1) high translation quality, (2) high match accuracy, and (3) low latency. In this work, we propose a template-based method that can yield results with high translation quality and match accuracy while keeping the decoding speed. Our basic idea is to rearrange the generation of constrained and unconstrained tokens through a template. The generation and derivation of the template can be learned through one sequence-to-sequence training framework. Thus our method does not require any changes in the model architecture and the decoding algorithm, making it easy to apply. Experimental results show that the proposed template-based methods can outperform several representative baselines in lexically and structurally constrained translation tasks.

Related papers

• Accelerating Transformer Inference for Translation via Parallel Decoding [2.89306442817912]
  Autoregressive decoding limits the efficiency of transformers for Machine Translation (MT) We present three parallel decoding algorithms and test them on different languages and models.
  arXiv  Detail & Related papers  (2023-05-17T17:57:34Z)
• Confident Adaptive Language Modeling [95.45272377648773]
  CALM is a framework for dynamically allocating different amounts of compute per input and generation timestep. We demonstrate the efficacy of our framework in reducing compute -- potential speedup of up to $times 3$ -- while provably maintaining high performance.
  arXiv  Detail & Related papers  (2022-07-14T17:00:19Z)
• Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization [94.4409074435894]
  We propose a novel and effective fine-tuning framework, named Layerwise Noise Stability Regularization (LNSR) Specifically, we propose to inject the standard Gaussian noise and regularize hidden representations of the fine-tuned model. We demonstrate the advantages of the proposed method over other state-of-the-art algorithms including L2-SP, Mixout and SMART.
  arXiv  Detail & Related papers  (2022-06-12T04:42:49Z)
• Distributionally Robust Multilingual Machine Translation [94.51866646879337]
  We propose a new learning objective for Multilingual neural machine translation (MNMT) based on distributionally robust optimization. We show how to practically optimize this objective for large translation corpora using an iterated best response scheme. Our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.
  arXiv  Detail & Related papers  (2021-09-09T03:48:35Z)
• Modelling Latent Translations for Cross-Lingual Transfer [47.61502999819699]
  We propose a new technique that integrates both steps of the traditional pipeline (translation and classification) into a single model. We evaluate our novel latent translation-based model on a series of multilingual NLU tasks. We report gains for both zero-shot and few-shot learning setups, up to 2.7 accuracy points on average.
  arXiv  Detail & Related papers  (2021-07-23T17:11:27Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Character-level Transformer-based Neural Machine Translation [5.699756532377753]
  We discuss a novel, Transformer-based approach, that we compare, both in speed and in quality to the Transformer at subword and character levels. We evaluate our models on 4 language pairs from WMT'15: DE-EN, CS-EN, FI-EN and RU-EN. The proposed novel architecture can be trained on a single GPU and is 34% percent faster than the character-level Transformer.
  arXiv  Detail & Related papers  (2020-05-22T15:40:43Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)
• Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders [37.949101113934226]
  Text simplification (TS) rephrases long sentences into simplified variants while preserving inherent semantics. This work investigates how to leverage large amounts of unpaired corpora in TS task. We propose asymmetric denoising methods for sentences with separate complexity.
  arXiv  Detail & Related papers  (2020-04-30T11:19:04Z)
• Lexically Constrained Neural Machine Translation with Levenshtein Transformer [8.831954614241234]
  This paper proposes a simple and effective algorithm for incorporating lexical constraints in neural machine translation. Our method injects terminology constraints at inference time without any impact on decoding speed.
  arXiv  Detail & Related papers  (2020-04-27T09:59:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.