Advancing STT for Low-Resource Real-World Speech

• URL: http://arxiv.org/abs/2506.08836v1
• Date: Tue, 10 Jun 2025 14:22:48 GMT
• Title: Advancing STT for Low-Resource Real-World Speech
• Authors: Flavio D'Intino, Hans-Peter Hutter,
• Abstract summary: This paper introduces the new SRB-300 dataset, a 300-hour annotated speech corpus.<n>It captures spontaneous speech across all major Swiss dialects recorded in various realistic environments.<n>We fine-tuned multiple OpenAI Whisper models on the SRB-300 dataset, achieving notable enhancements over previous zero-shot performance metrics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Swiss German is a low-resource language represented by diverse dialects that differ significantly from Standard German and from each other, lacking a standardized written form. As a result, transcribing Swiss German involves translating into Standard German. Existing datasets have been collected in controlled environments, yielding effective speech-to-text (STT) models, but these models struggle with spontaneous conversational speech. This paper, therefore, introduces the new SRB-300 dataset, a 300-hour annotated speech corpus featuring real-world long-audio recordings from 39 Swiss German radio and TV stations. It captures spontaneous speech across all major Swiss dialects recorded in various realistic environments and overcomes the limitation of prior sentence-level corpora. We fine-tuned multiple OpenAI Whisper models on the SRB-300 dataset, achieving notable enhancements over previous zero-shot performance metrics. Improvements in word error rate (WER) ranged from 19% to 33%, while BLEU scores increased between 8% and 40%. The best fine-tuned model, large-v3, achieved a WER of 17.1% and a BLEU score of 74.8. This advancement is crucial for developing effective and robust STT systems for Swiss German and other low-resource languages in real-world contexts.

Related papers

• Voice Adaptation for Swiss German [7.4162190889971]
  This work investigates the performance of Voice Adaptation models for Swiss German dialects, i.e., translating Standard German text to Swiss German dialect speech.<n>For this, we preprocess a large dataset of Swiss podcasts, which we automatically transcribe and annotate with dialect classes.<n>We fine-tune the XTTSv2 model on this dataset and show that it achieves good scores in human and automated evaluations and can correctly render the desired dialect.
  arXiv  Detail & Related papers  (2025-05-28T07:24:40Z)
• Fine-tuning Whisper on Low-Resource Languages for Real-World Applications [1.5908667698635532]
  Non-sentence-level data, which could improve the performance of long-form audio, is difficult to obtain and often restricted by copyright laws.<n>Our method bridges this gap by transforming more accessible sentence-level data into a format that preserves the model's ability to handle long-form audio.<n>We compare our model with a non-fine-tuned Whisper and our previous state-of-the-art Swiss German STT models, where our new model achieves higher BLEU scores.
  arXiv  Detail & Related papers  (2024-12-20T09:49:02Z)
• Whisper Finetuning on Nepali Language [0.0]
  This research focuses on making an exhaustive and generalized dataset followed by fine-tuning OpenAI's Whisper models to improve transcription accuracy for the Nepali language. We leverage publicly available ASR datasets and self-recorded custom datasets with a diverse range of accents, dialects, and speaking styles further enriched through augmentation. Our approach outperforms Whisper's baseline models trained on Fleur's dataset, achieving WER reductions of up to 36.2% on the small and 23.8% on medium models.
  arXiv  Detail & Related papers  (2024-11-19T15:55:56Z)
• Improving Multilingual ASR in the Wild Using Simple N-best Re-ranking [68.77659513993507]
  We present a simple and effective N-best re-ranking approach to improve multilingual ASR accuracy. Our results show spoken language identification accuracy improvements of 8.7% and 6.1%, respectively, and word error rates which are 3.3% and 2.0% lower on these benchmarks.
  arXiv  Detail & Related papers  (2024-09-27T03:31:32Z)
• Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR [74.38242498079627]
  Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable.<n>In this paper, SSL discrete speech features extracted from WavLM models are used as additional cross-utterance acoustic context features in Zipformer-Transducer ASR systems.
  arXiv  Detail & Related papers  (2024-09-13T13:01:09Z)
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation [90.71078166159295]
  We introduce SeamlessM4T, a single model that supports speech-to-speech translation, speech-to-text translation, text-to-text translation, and automatic speech recognition for up to 100 languages. We developed the first multilingual system capable of translating from and into English for both speech and text. On FLEURS, SeamlessM4T sets a new standard for translations into multiple target languages, achieving an improvement of 20% BLEU over the previous SOTA in direct speech-to-text translation.
  arXiv  Detail & Related papers  (2023-08-22T17:44:18Z)
• STT4SG-350: A Speech Corpus for All Swiss German Dialect Regions [5.6787416472329495]
  We present STT4SG-350 (Speech-to-Text for Swiss German), a corpus of Swiss German speech annotated with Standard German text at the sentence level. The data is collected using a web app in which the speakers are shown Standard German sentences, which they translate to Swiss German and record. It contains 343 hours of speech from all dialect regions and is the largest public speech corpus for Swiss German to date.
  arXiv  Detail & Related papers  (2023-05-30T08:49:38Z)
• Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages [76.95115818308918]
  We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the encoder of the model on a large unlabeled multilingual dataset of 12 million (M) hours spanning over 300 languages. We use multilingual pre-training with random-projection quantization and speech-text modality matching to achieve state-of-the-art performance on downstream multilingual ASR and speech-to-text translation tasks.
  arXiv  Detail & Related papers  (2023-03-02T07:47:18Z)
• No Language Left Behind: Scaling Human-Centered Machine Translation [69.28110770760506]
  We create datasets and models aimed at narrowing the performance gap between low and high-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-11T07:33:36Z)
• SDS-200: A Swiss German Speech to Standard German Text Corpus [5.370317759946287]
  We present SDS-200, a corpus of Swiss German dialectal speech with Standard German text translations. The data was collected using a web recording tool that is open to the public. The data consists of 200 hours of speech by around 4000 different speakers and covers a large part of the Swiss-German dialect landscape.
  arXiv  Detail & Related papers  (2022-05-19T12:16:29Z)
• Exploration of End-to-End ASR for OpenSTT -- Russian Open Speech-to-Text Dataset [73.66530509749305]
  This paper presents an exploration of end-to-end automatic speech recognition systems (ASR) for the largest open-source Russian language data set -- OpenSTT. We evaluate different existing end-to-end approaches such as joint CTC/Attention, RNN-Transducer, and Transformer. For the three available validation sets (phone calls, YouTube, and books), our best end-to-end model achieves word error rate (WER) of 34.8%, 19.1%, and 18.1%, respectively.
  arXiv  Detail & Related papers  (2020-06-15T10:35:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.