Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation

• URL: http://arxiv.org/abs/2012.14681v1
• Date: Tue, 29 Dec 2020 09:43:27 GMT
• Title: Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation
• Authors: Hyojung Han, Sathish Indurthi, Mohd Abbas Zaidi, Nikhil Kumar Lakumarapu, Beomseok Lee, Sangha Kim, Chanwoo Kim, Inchul Hwang
• Abstract summary: We propose a faster re-translation system based on a non-autoregressive sequence generation model (FReTNA) The proposed model reduces the average computation time by a factor of 20 when compared to the ReTA model. It also outperforms the streaming-based Wait-k model both in terms of time (1.5 times lower) and translation quality.
• Score: 10.773010211146694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, simultaneous translation has gathered a lot of attention since it enables compelling applications such as subtitle translation for a live event or real-time video-call translation. Some of these translation applications allow editing of partial translation giving rise to re-translation approaches. The current re-translation approaches are based on autoregressive sequence generation models (ReTA), which generate tar-get tokens in the (partial) translation sequentially. The multiple re-translations with sequential generation inReTAmodelslead to an increased inference time gap between the incoming source input and the corresponding target output as the source input grows. Besides, due to the large number of inference operations involved, the ReTA models are not favorable for resource-constrained devices. In this work, we propose a faster re-translation system based on a non-autoregressive sequence generation model (FReTNA) to overcome the aforementioned limitations. We evaluate the proposed model on multiple translation tasks and our model reduces the inference times by several orders and achieves a competitive BLEUscore compared to the ReTA and streaming (Wait-k) models.The proposed model reduces the average computation time by a factor of 20 when compared to the ReTA model by incurring a small drop in the translation quality. It also outperforms the streaming-based Wait-k model both in terms of computation time (1.5 times lower) and translation quality.

Related papers

• Efficient Machine Translation with a BiLSTM-Attention Approach [0.0]
  This paper proposes a novel Seq2Seq model aimed at improving translation quality while reducing the storage space required by the model. The model employs a Bidirectional Long Short-Term Memory network (Bi-LSTM) as the encoder to capture the context information of the input sequence. Compared to the current mainstream Transformer model, our model achieves superior performance on the WMT14 machine translation dataset.
  arXiv  Detail & Related papers  (2024-10-29T01:12:50Z)
• Non-autoregressive Sequence-to-Sequence Vision-Language Models [63.77614880533488]
  We propose a parallel decoding sequence-to-sequence vision-language model that marginalizes over multiple inference paths in the decoder. The model achieves performance on-par with its state-of-the-art autoregressive counterpart, but is faster at inference time.
  arXiv  Detail & Related papers  (2024-03-04T17:34:59Z)
• Exploring Linguistic Similarity and Zero-Shot Learning for Multilingual Translation of Dravidian Languages [0.34998703934432673]
  We build a single-decoder neural machine translation system for Dravidian-Dravidian multilingual translation. Our model achieves scores within 3 BLEU of large-scale pivot-based models when it is trained on 50% of the language directions.
  arXiv  Detail & Related papers  (2023-08-10T13:38:09Z)
• PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model [37.2192243883707]
  We propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation to generate fluent text. Results on semantic generation, text completion and summarization show its effectiveness in generating high-quality long-form text.
  arXiv  Detail & Related papers  (2023-06-05T01:36:39Z)
• Extrapolating Multilingual Understanding Models as Multilingual Generators [82.1355802012414]
  This paper explores methods to empower multilingual understanding models the generation abilities to get a unified model. We propose a textbfSemantic-textbfGuided textbfAlignment-then-Denoising (SGA) approach to adapt an encoder to a multilingual generator with a small number of new parameters.
  arXiv  Detail & Related papers  (2023-05-22T15:33:21Z)
• Modeling Context With Linear Attention for Scalable Document-Level Translation [72.41955536834702]
  We investigate the efficacy of a recent linear attention model on document translation and augment it with a sentential gate to promote a recency inductive bias. We show that sentential gating further improves translation quality on IWSLT.
  arXiv  Detail & Related papers  (2022-10-16T03:41:50Z)
• Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages [86.08359401867577]
  Back-translation is widely known for its effectiveness for neural machine translation when little to no parallel data is available. We propose performing back-translation via code summarization and generation. We show that our proposed approach performs competitively with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-23T08:20:41Z)
• Non-Autoregressive Neural Machine Translation: A Call for Clarity [3.1447111126465]
  We take a step back and revisit several techniques that have been proposed for improving non-autoregressive translation models. We provide novel insights for establishing strong baselines using length prediction or CTC-based architecture variants. We contribute standardized BLEU, chrF++, and TER scores using sacreBLEU on four translation tasks.
  arXiv  Detail & Related papers  (2022-05-21T12:15:22Z)
• Anticipation-free Training for Simultaneous Translation [70.85761141178597]
  Simultaneous translation (SimulMT) speeds up the translation process by starting to translate before the source sentence is completely available. Existing methods increase latency or introduce adaptive read-write policies for SimulMT models to handle local reordering and improve translation quality. We propose a new framework that decomposes the translation process into the monotonic translation step and the reordering step.
  arXiv  Detail & Related papers  (2022-01-30T16:29:37Z)
• Fast Sequence Generation with Multi-Agent Reinforcement Learning [40.75211414663022]
  Non-autoregressive decoding has been proposed in machine translation to speed up the inference time by generating all words in parallel. We propose a simple and efficient model for Non-Autoregressive sequence Generation (NAG) with a novel training paradigm: Counterfactuals-critical Multi-Agent Learning (CMAL) On MSCOCO image captioning benchmark, our NAG method achieves a performance comparable to state-of-the-art autoregressive models, while brings 13.9x decoding speedup.
  arXiv  Detail & Related papers  (2021-01-24T12:16:45Z)
• SimulEval: An Evaluation Toolkit for Simultaneous Translation [59.02724214432792]
  Simultaneous translation on both text and speech focuses on a real-time and low-latency scenario. SimulEval is an easy-to-use and general evaluation toolkit for both simultaneous text and speech translation.
  arXiv  Detail & Related papers  (2020-07-31T17:44:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.