ESPnet-ST IWSLT 2021 Offline Speech Translation System

• URL: http://arxiv.org/abs/2107.00636v1
• Date: Thu, 1 Jul 2021 17:49:43 GMT
• Title: ESPnet-ST IWSLT 2021 Offline Speech Translation System
• Authors: Hirofumi Inaguma, Brian Yan, Siddharth Dalmia, Pengcheng Gu, Jiatong Shi, Kevin Duh, Shinji Watanabe
• Abstract summary: This paper describes the ESPnet-ST group's IWSLT 2021 submission in the offline speech translation track. This year we made various efforts on training data, architecture, and audio segmentation. Our best E2E system combined all the techniques with model ensembling and achieved 31.4 BLEU.
• Score: 56.83606198051871
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes the ESPnet-ST group's IWSLT 2021 submission in the offline speech translation track. This year we made various efforts on training data, architecture, and audio segmentation. On the data side, we investigated sequence-level knowledge distillation (SeqKD) for end-to-end (E2E) speech translation. Specifically, we used multi-referenced SeqKD from multiple teachers trained on different amounts of bitext. On the architecture side, we adopted the Conformer encoder and the Multi-Decoder architecture, which equips dedicated decoders for speech recognition and translation tasks in a unified encoder-decoder model and enables search in both source and target language spaces during inference. We also significantly improved audio segmentation by using the pyannote.audio toolkit and merging multiple short segments for long context modeling. Experimental evaluations showed that each of them contributed to large improvements in translation performance. Our best E2E system combined all the above techniques with model ensembling and achieved 31.4 BLEU on the 2-ref of tst2021 and 21.2 BLEU and 19.3 BLEU on the two single references of tst2021.

Related papers

• KIT's Multilingual Speech Translation System for IWSLT 2023 [58.5152569458259]
  We describe our speech translation system for the multilingual track of IWSLT 2023. The task requires translation into 10 languages of varying amounts of resources. Our cascaded speech system substantially outperforms its end-to-end counterpart on scientific talk translation.
  arXiv  Detail & Related papers  (2023-06-08T16:13:20Z)
• Improved Cross-Lingual Transfer Learning For Automatic Speech Translation [18.97234151624098]
  We show that by initializing the encoder of the encoder-decoder sequence-to-sequence translation model with SAMU-XLS-R, we achieve significantly better cross-lingual task knowledge transfer. We demonstrate the effectiveness of our approach on two popular datasets, namely, CoVoST-2 and Europarl.
  arXiv  Detail & Related papers  (2023-06-01T15:19:06Z)
• The YiTrans End-to-End Speech Translation System for IWSLT 2022 Offline Shared Task [92.5087402621697]
  This paper describes the submission of our end-to-end YiTrans speech translation system for the IWSLT 2022 offline task. The YiTrans system is built on large-scale pre-trained encoder-decoder models. Our final submissions rank first on English-German and English-Chinese end-to-end systems in terms of the automatic evaluation metric.
  arXiv  Detail & Related papers  (2022-06-12T16:13:01Z)
• The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task [23.008938777422767]
  This paper describes the submission of the NiuTrans end-to-end speech translation system for the IWSLT 2021 offline task. We use the Transformer-based model architecture and enhance it by Conformer, relative position encoding, and stacked acoustic and textual encoding. We achieve 33.84 BLEU points on the MuST-C En-De test set, which shows the enormous potential of the end-to-end model.
  arXiv  Detail & Related papers  (2021-07-06T07:45:23Z)
• Multilingual Speech Translation with Unified Transformer: Huawei Noah's Ark Lab at IWSLT 2021 [33.876412404781846]
  This paper describes the system submitted to the IWSLT 2021 Speech Translation (MultiST) task from Huawei Noah's Ark Lab. We use a unified transformer architecture for our MultiST model, so that the data from different modalities can be exploited to enhance the model's ability. We apply several training techniques to improve the performance, including multi-task learning, task-level curriculum learning, data augmentation, etc.
  arXiv  Detail & Related papers  (2021-06-01T02:50:49Z)
• UPC's Speech Translation System for IWSLT 2021 [2.099922236065961]
  This paper describes the submission to the IWSLT 2021 offline speech translation task by the UPC Machine Translation group. The task consists of building a system capable of translating English audio recordings extracted from TED talks into German text. Our submission is an end-to-end speech translation system, which combines pre-trained models with coupling modules between the encoder and decoder.
  arXiv  Detail & Related papers  (2021-05-10T17:04:11Z)
• Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation [71.54816893482457]
  We introduce dual-decoder Transformer, a new model architecture that jointly performs automatic speech recognition (ASR) and multilingual speech translation (ST) Our models are based on the original Transformer architecture but consist of two decoders, each responsible for one task (ASR or ST)
  arXiv  Detail & Related papers  (2020-11-02T04:59:50Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)
• Bi-Decoder Augmented Network for Neural Machine Translation [108.3931242633331]
  We propose a novel Bi-Decoder Augmented Network (BiDAN) for the neural machine translation task. Since each decoder transforms the representations of the input text into its corresponding language, jointly training with two target ends can make the shared encoder has the potential to produce a language-independent semantic space.
  arXiv  Detail & Related papers  (2020-01-14T02:05:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.