IsometricMT: Neural Machine Translation for Automatic Dubbing

• URL: http://arxiv.org/abs/2112.08682v1
• Date: Thu, 16 Dec 2021 08:03:20 GMT
• Title: IsometricMT: Neural Machine Translation for Automatic Dubbing
• Authors: Surafel M. Lakew, Yogesh Virkar, Prashant Mathur, Marcello Federico
• Abstract summary: This work introduces a self-learning approach that allows a transformer model to directly learn to generate outputs that closely match the source length. We report results on four language pairs with a publicly available benchmark based on TED Talk data.
• Score: 9.605781943224251
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic dubbing (AD) is among the use cases where translations should fit a given length template in order to achieve synchronicity between source and target speech. For neural machine translation (MT), generating translations of length close to the source length (e.g. within +-10% in character count), while preserving quality is a challenging task. Controlling NMT output length comes at a cost to translation quality which is usually mitigated with a two step approach of generation of n-best hypotheses and then re-ranking them based on length and quality. This work, introduces a self-learning approach that allows a transformer model to directly learn to generate outputs that closely match the source length, in short isometric MT. In particular, our approach for isometric MT does not require to generate multiple hypotheses nor any auxiliary scoring function. We report results on four language pairs (English - French, Italian, German, Spanish) with a publicly available benchmark based on TED Talk data. Both automatic and manual evaluations show that our self-learning approach to performs on par with more complex isometric MT approaches.

Related papers

• Towards Zero-Shot Multimodal Machine Translation [64.9141931372384]
  We propose a method to bypass the need for fully supervised data to train multimodal machine translation systems. Our method, called ZeroMMT, consists in adapting a strong text-only machine translation (MT) model by training it on a mixture of two objectives. To prove that our method generalizes to languages with no fully supervised training data available, we extend the CoMMuTE evaluation dataset to three new languages: Arabic, Russian and Chinese.
  arXiv  Detail & Related papers  (2024-07-18T15:20:31Z)
• Back Translation for Speech-to-text Translation Without Transcripts [11.13240570688547]
  We develop a back translation algorithm for ST (BT4ST) to synthesize pseudo ST data from monolingual target data. To ease the challenges posed by short-to-long generation and one-to-many mapping, we introduce self-supervised discrete units. With our synthetic ST data, we achieve an average boost of 2.3 BLEU on MuST-C En-De, En-Fr, and En-Es datasets.
  arXiv  Detail & Related papers  (2023-05-15T15:12:40Z)
• Exploiting Language Relatedness in Machine Translation Through Domain Adaptation Techniques [3.257358540764261]
  We present a novel approach of using a scaled similarity score of sentences, especially for related languages based on a 5-gram KenLM language model. Our approach succeeds in increasing 2 BLEU point on multi-domain approach, 3 BLEU point on fine-tuning for NMT and 2 BLEU point on iterative back-translation approach.
  arXiv  Detail & Related papers  (2023-03-03T09:07:30Z)
• Efficient Inference for Multilingual Neural Machine Translation [60.10996883354372]
  We consider several ways to make multilingual NMT faster at inference without degrading its quality. Our experiments demonstrate that combining a shallow decoder with vocabulary filtering leads to more than twice faster inference with no loss in translation quality.
  arXiv  Detail & Related papers  (2021-09-14T13:28:13Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation [88.78138830698173]
  We focus on sequence-level knowledge distillation (SeqKD) from external text-based NMT models. We train a bilingual E2E-ST model to predict paraphrased transcriptions as an auxiliary task with a single decoder.
  arXiv  Detail & Related papers  (2021-04-13T19:00:51Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing [11.564158965143418]
  We frame the task of machine translation evaluation as one of scoring machine translation output with a sequence-to-sequence paraphraser. We propose training the paraphraser as a multilingual NMT system, treating paraphrasing as a zero-shot translation task. Our method is simple and intuitive, and does not require human judgements for training.
  arXiv  Detail & Related papers  (2020-04-30T03:32:34Z)
• Bootstrapping a Crosslingual Semantic Parser [74.99223099702157]
  We adapt a semantic trained on a single language, such as English, to new languages and multiple domains with minimal annotation. We query if machine translation is an adequate substitute for training data, and extend this to investigate bootstrapping using joint training with English, paraphrasing, and multilingual pre-trained models.
  arXiv  Detail & Related papers  (2020-04-06T12:05:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.