HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translation

• URL: http://arxiv.org/abs/2306.11252v1
• Date: Tue, 20 Jun 2023 03:09:32 GMT
• Title: HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translation
• Authors: Cihan Xiao, Henry Li Xinyuan, Jinyi Yang, Dongji Gao, Matthew Wiesner, Kevin Duh, Sanjeev Khudanpur
• Abstract summary: We introduce HK-LegiCoST, a new three-way parallel corpus of Cantonese-English translations. We describe the notable challenges in corpus preparation: segmentation, alignment of long audio recordings, and sentence-level alignment with non-verbatim transcripts.
• Score: 29.990957948085956
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce HK-LegiCoST, a new three-way parallel corpus of Cantonese-English translations, containing 600+ hours of Cantonese audio, its standard traditional Chinese transcript, and English translation, segmented and aligned at the sentence level. We describe the notable challenges in corpus preparation: segmentation, alignment of long audio recordings, and sentence-level alignment with non-verbatim transcripts. Such transcripts make the corpus suitable for speech translation research when there are significant differences between the spoken and written forms of the source language. Due to its large size, we are able to demonstrate competitive speech translation baselines on HK-LegiCoST and extend them to promising cross-corpus results on the FLEURS Cantonese subset. These results deliver insights into speech recognition and translation research in languages for which non-verbatim or ``noisy'' transcription is common due to various factors, including vernacular and dialectal speech.

Related papers

• What Makes a Good Speech Tokenizer for LLM-Centric Speech Generation? A Systematic Study [58.55905182336196]
  Speech-language models (SLMs) offer a promising path toward unifying speech and text understanding and generation.<n>We investigate the role of speech tokenizer designs in LLM-centric SLMs, augmented by speech heads and speaker modeling.<n>We introduce multi-token prediction (MTP) into SLMs, enabling each hidden state to decode multiple speech tokens.
  arXiv  Detail & Related papers  (2025-06-14T15:26:31Z)
• Low-Resource NMT: A Case Study on the Written and Spoken Languages in Hong Kong [25.358712649791393]
  Spoken Cantonese can be transcribed into Chinese characters, which constitute the so-called written Cantonese.<n>Written Cantonese exhibits significant lexical and grammatical differences from standard written Chinese.<n>This paper describes a transformer-based neural machine translation (NMT) system for written-Chinese-to-written-Cantonese translation.
  arXiv  Detail & Related papers  (2025-05-23T12:32:01Z)
• Connecting Voices: LoReSpeech as a Low-Resource Speech Parallel Corpus [0.0]
  This paper introduces a methodology for constructing LoReSpeech, a low-resource speech-to-speech translation corpus. LoReSpeech delivers both intra- and inter-language alignments, enabling advancements in multilingual ASR systems.
  arXiv  Detail & Related papers  (2025-02-25T14:00:15Z)
• Cross-Lingual Transfer Learning for Speech Translation [7.802021866251242]
  This paper examines how to expand the speech translation capability of speech foundation models with restricted data. Whisper, a speech foundation model with strong performance on speech recognition and English translation, is used as the example model. Using speech-to-speech retrieval to analyse the audio representations generated by the encoder, we show that utterances from different languages are mapped to a shared semantic space.
  arXiv  Detail & Related papers  (2024-07-01T09:51:48Z)
• TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation [97.54885207518946]
  We introduce a novel model framework TransVIP that leverages diverse datasets in a cascade fashion. We propose two separated encoders to preserve the speaker's voice characteristics and isochrony from the source speech during the translation process. Our experiments on the French-English language pair demonstrate that our model outperforms the current state-of-the-art speech-to-speech translation model.
  arXiv  Detail & Related papers  (2024-05-28T04:11:37Z)
• Enhancing Cross-lingual Transfer via Phonemic Transcription Integration [57.109031654219294]
  PhoneXL is a framework incorporating phonemic transcriptions as an additional linguistic modality for cross-lingual transfer. Our pilot study reveals phonemic transcription provides essential information beyond the orthography to enhance cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-07-10T06:17:33Z)
• VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbing [73.56970726406274]
  Video dubbing aims to translate the original speech in a film or television program into the speech in a target language. To ensure the translated speech to be well aligned with the corresponding video, the length/duration of the translated speech should be as close as possible to that of the original speech. We propose a machine translation system tailored for the task of video dubbing, which directly considers the speech duration of each token in translation.
  arXiv  Detail & Related papers  (2022-11-30T12:09:40Z)
• BSTC: A Large-Scale Chinese-English Speech Translation Dataset [26.633433687767553]
  BSTC (Baidu Speech Translation Corpus) is a large-scale Chinese-English speech translation dataset. This dataset is constructed based on a collection of licensed videos of talks or lectures, including about 68 hours of Mandarin data. We have asked three experienced interpreters to simultaneously interpret the testing talks in a mock conference setting.
  arXiv  Detail & Related papers  (2021-04-08T07:38:51Z)
• The Multilingual TEDx Corpus for Speech Recognition and Translation [30.993199499048824]
  We present the Multilingual TEDx corpus, built to support speech recognition (ASR) and speech translation (ST) research across many non-English source languages. The corpus is a collection of audio recordings from TEDx talks in 8 source languages. We segment transcripts into sentences and align them to the source-language audio and target-language translations.
  arXiv  Detail & Related papers  (2021-02-02T21:16:25Z)
• Bridging the Modality Gap for Speech-to-Text Translation [57.47099674461832]
  End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way. Most existing methods employ an encoder-decoder structure with a single encoder to learn acoustic representation and semantic information simultaneously. We propose a Speech-to-Text Adaptation for Speech Translation model which aims to improve the end-to-end model performance by bridging the modality gap between speech and text.
  arXiv  Detail & Related papers  (2020-10-28T12:33:04Z)
• Consecutive Decoding for Speech-to-text Translation [51.155661276936044]
  COnSecutive Transcription and Translation (COSTT) is an integral approach for speech-to-text translation. The key idea is to generate source transcript and target translation text with a single decoder. Our method is verified on three mainstream datasets.
  arXiv  Detail & Related papers  (2020-09-21T10:10:45Z)
• "Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text Translation [49.610188741500274]
  An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel corpus. We build a system to fully utilize signals in a parallel ST corpus.
  arXiv  Detail & Related papers  (2020-09-21T09:19:07Z)
• FT Speech: Danish Parliament Speech Corpus [21.190182627955817]
  This paper introduces FT Speech, a new speech corpus created from the recorded meetings of the Danish Parliament. The corpus contains over 1,800 hours of transcribed speech by a total of 434 speakers. It is significantly larger in duration, vocabulary, and amount of spontaneous speech than the existing public speech corpora for Danish.
  arXiv  Detail & Related papers  (2020-05-25T19:51:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.