Does Simultaneous Speech Translation need Simultaneous Models?

• URL: http://arxiv.org/abs/2204.03783v1
• Date: Fri, 8 Apr 2022 00:10:46 GMT
• Title: Does Simultaneous Speech Translation need Simultaneous Models?
• Authors: Sara Papi, Marco Gaido, Matteo Negri, Marco Turchi
• Abstract summary: We show that a single model trained offline can effectively serve not only offline but also simultaneous tasks at different latency regimes. This single-model solution does not only facilitate the adoption of well-established offline techniques and architectures without affecting latency but also yields similar or even better translation quality compared to the same model trained in the simultaneous setting.
• Score: 17.305879157385675
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task. To meet the latency constraints posed by different application scenarios, multiple dedicated SimulST models are usually trained and maintained, causing high computational costs and increased environmental impact. In this paper, we show that a single model trained offline can effectively serve not only offline but also simultaneous tasks at different latency regimes, bypassing any training/adaptation procedures. This single-model solution does not only facilitate the adoption of well-established offline techniques and architectures without affecting latency but also yields similar or even better translation quality compared to the same model trained in the simultaneous setting. Experiments on En$\rightarrow$\{De, Es\} indicate the effectiveness of our approach, showing competitive results with the SimulST state of the art.

Related papers

• Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters [5.312303275762104]
  Simultaneous machine translation aims at solving the task of real-time translation by starting to translate before consuming the full input. wait-$k$ policy offers a solution by starting to translate after consuming $k$ words. In this paper, we address the challenge of building one model that can fulfil multiple latency levels.
  arXiv  Detail & Related papers  (2024-07-18T12:42:45Z)
• Simultaneous Machine Translation with Large Language Models [51.470478122113356]
  We investigate the possibility of applying Large Language Models to SimulMT tasks. We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
  arXiv  Detail & Related papers  (2023-09-13T04:06:47Z)
• Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data [16.05089716626287]
  We propose an effective way to train a SimulST model using mixed data of SI and offline. Experiment results show improvements of BLEURT in different latency ranges.
  arXiv  Detail & Related papers  (2023-06-14T15:42:06Z)
• eP-ALM: Efficient Perceptual Augmentation of Language Models [70.47962271121389]
  We propose to direct effort to efficient adaptations of existing models, and propose to augment Language Models with perception. Existing approaches for adapting pretrained models for vision-language tasks still rely on several key components that hinder their efficiency. We show that by freezing more than 99% of total parameters, training only one linear projection layer, and prepending only one trainable token, our approach (dubbed eP-ALM) significantly outperforms other baselines on VQA and Captioning.
  arXiv  Detail & Related papers  (2023-03-20T19:20:34Z)
• Planning for Sample Efficient Imitation Learning [52.44953015011569]
  Current imitation algorithms struggle to achieve high performance and high in-environment sample efficiency simultaneously. We propose EfficientImitate, a planning-based imitation learning method that can achieve high in-environment sample efficiency and performance simultaneously. Experimental results show that EI achieves state-of-the-art results in performance and sample efficiency.
  arXiv  Detail & Related papers  (2022-10-18T05:19:26Z)
• Data-Driven Adaptive Simultaneous Machine Translation [51.01779863078624]
  We propose a novel and efficient training scheme for adaptive SimulMT. Our method outperforms all strong baselines in terms of translation quality and latency.
  arXiv  Detail & Related papers  (2022-04-27T02:40:21Z)
• Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy [6.487736084189248]
  Simultaneous machine translation (SiMT) generates translation before reading the entire source sentence. Previous methods usually need to train multiple SiMT models for different latency levels, resulting in large computational costs. We propose a universal SiMT model with Mixture-of-Experts Wait-k Policy to achieve the best translation quality under arbitrary latency.
  arXiv  Detail & Related papers  (2021-09-11T09:43:15Z)
• Multi-mode Transformer Transducer with Stochastic Future Context [53.005638503544866]
  Multi-mode speech recognition models can process longer future context to achieve higher accuracy and when a latency budget is not flexible, the model can still achieve reliable accuracy. We show that a Multi-mode ASR model rivals, if not surpasses, a set of competitive streaming baselines trained with different latency budgets.
  arXiv  Detail & Related papers  (2021-06-17T18:42:11Z)
• SimulEval: An Evaluation Toolkit for Simultaneous Translation [59.02724214432792]
  Simultaneous translation on both text and speech focuses on a real-time and low-latency scenario. SimulEval is an easy-to-use and general evaluation toolkit for both simultaneous text and speech translation.
  arXiv  Detail & Related papers  (2020-07-31T17:44:41Z)
• FastBERT: a Self-distilling BERT with Adaptive Inference Time [14.207970599841218]
  Our model achieves promising results in twelve English and Chinese datasets. It is able to speed up by a wide range from 1 to 12 times than BERT if given different speedup thresholds.
  arXiv  Detail & Related papers  (2020-04-05T12:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.