Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data

• URL: http://arxiv.org/abs/2306.08582v1
• Date: Wed, 14 Jun 2023 15:42:06 GMT
• Title: Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data
• Authors: Yuka Ko, Ryo Fukuda, Yuta Nishikawa, Yasumasa Kano, Katsuhito Sudoh, Satoshi Nakamura
• Abstract summary: We propose an effective way to train a SimulST model using mixed data of SI and offline. Experiment results show improvements of BLEURT in different latency ranges.
• Score: 16.05089716626287
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Simultaneous speech translation (SimulST) translates partial speech inputs incrementally. Although the monotonic correspondence between input and output is preferable for smaller latency, it is not the case for distant language pairs such as English and Japanese. A prospective approach to this problem is to mimic simultaneous interpretation (SI) using SI data to train a SimulST model. However, the size of such SI data is limited, so the SI data should be used together with ordinary bilingual data whose translations are given in offline. In this paper, we propose an effective way to train a SimulST model using mixed data of SI and offline. The proposed method trains a single model using the mixed data with style tags that tell the model to generate SI- or offline-style outputs. Experiment results show improvements of BLEURT in different latency ranges, and our analyses revealed the proposed model generates SI-style outputs more than the baseline.

Related papers

• Word Order in English-Japanese Simultaneous Interpretation: Analyses and Evaluation using Chunk-wise Monotonic Translation [13.713981533436135]
  This paper analyzes the features of monotonic translations, which follow the word order of the source language, in simultaneous interpreting (SI) We analyzed the characteristics of chunk-wise monotonic translation (CMT) sentences using the NAIST English-to-Japanese Chunk-wise Monotonic Translation Evaluation dataset. We further investigated the features of CMT sentences by evaluating the output from the existing speech translation (ST) and simultaneous speech translation (simulST) models on the NAIST English-to-Japanese Chunk-wise Monotonic Translation Evaluation dataset.
  arXiv  Detail & Related papers  (2024-06-13T09:10:16Z)
• Simultaneous Machine Translation with Large Language Models [51.470478122113356]
  We investigate the possibility of applying Large Language Models to SimulMT tasks. We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
  arXiv  Detail & Related papers  (2023-09-13T04:06:47Z)
• NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpus [23.49376007047965]
  It remains a question that how simultaneous interpretation (SI) data affects simultaneous machine translation (SiMT) We introduce NAIST-SIC-Aligned, which is an automatically-aligned parallel English-Japanese SI dataset. Our results show that models trained with SI data lead to significant improvement in translation quality and latency over baselines.
  arXiv  Detail & Related papers  (2023-04-23T23:03:58Z)
• Improving Simultaneous Machine Translation with Monolingual Data [94.1085601198393]
  Simultaneous machine translation (SiMT) is usually done via sequence-level knowledge distillation (Seq-KD) from a full-sentence neural machine translation (NMT) model. We propose to leverage monolingual data to improve SiMT, which trains a SiMT student on the combination of bilingual data and external monolingual data distilled by Seq-KD.
  arXiv  Detail & Related papers  (2022-12-02T14:13:53Z)
• Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation [49.916963624249355]
  A UNMT model is trained on the pseudo parallel data with translated source, and natural source sentences in inference. The source discrepancy between training and inference hinders the translation performance of UNMT models. We propose an online self-training approach, which simultaneously uses the pseudo parallel data natural source, translated target to mimic the inference scenario.
  arXiv  Detail & Related papers  (2022-03-16T04:50:27Z)
• Anticipation-free Training for Simultaneous Translation [70.85761141178597]
  Simultaneous translation (SimulMT) speeds up the translation process by starting to translate before the source sentence is completely available. Existing methods increase latency or introduce adaptive read-write policies for SimulMT models to handle local reordering and improve translation quality. We propose a new framework that decomposes the translation process into the monotonic translation step and the reordering step.
  arXiv  Detail & Related papers  (2022-01-30T16:29:37Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer [33.876412404781846]
  RealTranS is an end-to-end model for simultaneous speech translation. It maps speech features into text space with a weighted-shrinking operation and a semantic encoder. Experiments show that RealTranS with the Wait-K-Stride-N strategy outperforms prior end-to-end models.
  arXiv  Detail & Related papers  (2021-06-09T06:35:46Z)
• Cross-language Sentence Selection via Data Augmentation and Rationale Training [22.106577427237635]
  It uses data augmentation and negative sampling techniques on noisy parallel sentence data to learn a cross-lingual embedding-based query relevance model. Results show that this approach performs as well as or better than multiple state-of-the-art machine translation + monolingual retrieval systems trained on the same parallel data.
  arXiv  Detail & Related papers  (2021-06-04T07:08:47Z)
• Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation [98.83925811122795]
  We propose to improve the sampling procedure by selecting the most informative monolingual sentences to complement the parallel data. We compute the uncertainty of monolingual sentences using the bilingual dictionary extracted from the parallel data. Experimental results on large-scale WMT English$Rightarrow$German and English$Rightarrow$Chinese datasets demonstrate the effectiveness of the proposed approach.
  arXiv  Detail & Related papers  (2021-06-02T05:01:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.