Dialectal Speech Recognition and Translation of Swiss German Speech to Standard German Text: Microsoft's Submission to SwissText 2021

• URL: http://arxiv.org/abs/2106.08126v1
• Date: Tue, 15 Jun 2021 13:34:02 GMT
• Title: Dialectal Speech Recognition and Translation of Swiss German Speech to Standard German Text: Microsoft's Submission to SwissText 2021
• Authors: Yuriy Arabskyy, Aashish Agarwal, Subhadeep Dey, Oscar Koller
• Abstract summary: Swiss German refers to the multitude of Alemannic dialects spoken in the German-speaking parts of Switzerland. We propose a hybrid automatic speech recognition system with a lexicon that incorporates translations. Our submission reaches 46.04% BLEU on a blind conversational test set and outperforms the second best competitor by a 12% relative margin.
• Score: 17.675379299410054
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes the winning approach in the public SwissText 2021 competition on dialect recognition and translation of Swiss German speech to standard German text. Swiss German refers to the multitude of Alemannic dialects spoken in the German-speaking parts of Switzerland. Swiss German differs significantly from standard German in pronunciation, word inventory and grammar. It is mostly incomprehensible to native German speakers. Moreover, it lacks a standardized written script. To solve the challenging task, we propose a hybrid automatic speech recognition system with a lexicon that incorporates translations, a 1st pass language model that deals with Swiss German particularities, a transfer-learned acoustic model and a strong neural language model for 2nd pass rescoring. Our submission reaches 46.04% BLEU on a blind conversational test set and outperforms the second best competitor by a 12% relative margin.

Related papers

• Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect [52.1701152610258]
  Adding a Swiss German adapter to a modular encoder achieves 97.5% of fully monolithic adaptation performance. For the task of retrieving Swiss German sentences given Standard German queries, adapting a character-level model is more effective than the other adaptation strategies.
  arXiv  Detail & Related papers  (2024-01-25T18:59:32Z)
• Dialect Transfer for Swiss German Speech Translation [9.373232685350844]
  This paper investigates the challenges in building Swiss German speech translation systems. It focuses on the impact of dialect diversity and differences between Swiss German and Standard German.
  arXiv  Detail & Related papers  (2023-10-13T13:16:57Z)
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation [90.71078166159295]
  We introduce SeamlessM4T, a single model that supports speech-to-speech translation, speech-to-text translation, text-to-text translation, and automatic speech recognition for up to 100 languages. We developed the first multilingual system capable of translating from and into English for both speech and text. On FLEURS, SeamlessM4T sets a new standard for translations into multiple target languages, achieving an improvement of 20% BLEU over the previous SOTA in direct speech-to-text translation.
  arXiv  Detail & Related papers  (2023-08-22T17:44:18Z)
• STT4SG-350: A Speech Corpus for All Swiss German Dialect Regions [5.6787416472329495]
  We present STT4SG-350 (Speech-to-Text for Swiss German), a corpus of Swiss German speech annotated with Standard German text at the sentence level. The data is collected using a web app in which the speakers are shown Standard German sentences, which they translate to Swiss German and record. It contains 343 hours of speech from all dialect regions and is the largest public speech corpus for Swiss German to date.
  arXiv  Detail & Related papers  (2023-05-30T08:49:38Z)
• ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation [79.66359274050885]
  We present ComSL, a speech-language model built atop a composite architecture of public pretrained speech-only and language-only models. Our approach has demonstrated effectiveness in end-to-end speech-to-text translation tasks.
  arXiv  Detail & Related papers  (2023-05-24T07:42:15Z)
• Scaling Speech Technology to 1,000+ Languages [66.31120979098483]
  The Massively Multilingual Speech (MMS) project increases the number of supported languages by 10-40x, depending on the task. Main ingredients are a new dataset based on readings of publicly available religious texts. We built pre-trained wav2vec 2.0 models covering 1,406 languages, a single multilingual automatic speech recognition model for 1,107 languages, speech synthesis models for the same number of languages, and a language identification model for 4,017 languages.
  arXiv  Detail & Related papers  (2023-05-22T22:09:41Z)
• SwissBERT: The Multilingual Language Model for Switzerland [52.1701152610258]
  SwissBERT is a masked language model created specifically for processing Switzerland-related text. SwissBERT is a pre-trained model that we adapted to news articles written in the national languages of Switzerland. Since SwissBERT uses language adapters, it may be extended to Swiss German dialects in future work.
  arXiv  Detail & Related papers  (2023-03-23T14:44:47Z)
• 2nd Swiss German Speech to Standard German Text Shared Task at SwissText 2022 [3.910747992453137]
  The objective was to maximize the BLEU score on a test set of Grisons speech. 3 teams participated, with the best-performing system achieving a BLEU score of 70.1.
  arXiv  Detail & Related papers  (2023-01-17T10:31:11Z)
• SDS-200: A Swiss German Speech to Standard German Text Corpus [5.370317759946287]
  We present SDS-200, a corpus of Swiss German dialectal speech with Standard German text translations. The data was collected using a web recording tool that is open to the public. The data consists of 200 hours of speech by around 4000 different speakers and covers a large part of the Swiss-German dialect landscape.
  arXiv  Detail & Related papers  (2022-05-19T12:16:29Z)
• A Swiss German Dictionary: Variation in Speech and Writing [45.82374977939355]
  We introduce a dictionary containing forms of common words in various Swiss German dialects normalized into High German. To alleviate the uncertainty associated with this diversity, we complement the pairs of Swiss German - High German words with the Swiss German phonetic transcriptions (SAMPA) This dictionary becomes thus the first resource to combine large-scale spontaneous translation with phonetic transcriptions.
  arXiv  Detail & Related papers  (2020-03-31T22:10:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.