Pashto Common Voice: Building the First Open Speech Corpus for a 60-Million-Speaker Low-Resource Language

• URL: http://arxiv.org/abs/2603.27021v1
• Date: Fri, 27 Mar 2026 22:22:03 GMT
• Title: Pashto Common Voice: Building the First Open Speech Corpus for a 60-Million-Speaker Low-Resource Language
• Abstract summary: We present the Pashto Common Voice corpus -- the first large-scale, openly licensed speech resource for Pashto.<n>Through a community effort spanning 2022-2025, the corpus grew from 1.5 hours and 5 contributors to 147 total hours and 1,483 unique speakers.<n>Speaker participation increased approximately 108-fold between CV17 and CV18, coinciding with a VOA Pashto broadcast campaign.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present the Pashto Common Voice corpus -- the first large-scale, openly licensed speech resource for Pashto, a language with over 60 million native speakers largely absent from open speech technology. Through a community effort spanning 2022-2025, the corpus grew from 1.5 hours and 5 contributors to 147 total hours and 1,483 unique speakers across ten Mozilla Common Voice releases (CV14-CV23). Speaker participation increased approximately 108-fold between CV17 and CV18, coinciding with a VOA Pashto broadcast campaign. We describe the full methodology: interface localisation, Wikipedia-based sentence extraction with automated filtering, phonemically targeted contributions for the four most frequently dropped Pashto characters, and multi-channel community outreach. MCV23 contains 107,781 clips (60,337 validated; 82.33 validated hours) across 13 content domains. Fine-tuning Whisper Base on the MCV20 yields 13.4% WER on the MCV20 test split, against the published Whisper Base zero-shot WER of 99.0% on Pashto.

Related papers

• BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language [0.20771014670599888]
  We present BaltiVoice, a 16.8-hour read-speech corpus for Balti (ISO 639-3: bft), a Tibetic language spoken in Gilgit-Baltistan, Pakistan.<n>The corpus contains 10,060 validated utterances in native Nastaliq script, derived from Mozilla Common Voice recordings.<n>Fine-tuning OpenAI Whisper-small yields a Word Error Rate (WER) of 26.74% and a Character Error Rate (CER) of 8.67% on a 538-utterance speaker-disjoint validation set.
  arXiv  Detail & Related papers  (2026-06-02T11:23:49Z)
• WorldSpeech: A Multilingual Speech Corpus from Around the World [35.61634772862795]
  WorldSpeech is a 24 kHz multilingual speech corpus comprising 65k hours of aligned audio-transcript data across 76 languages.<n>For 37 languages, WorldSpeech provides more than 200 hours of aligned speech, with 28 exceeding 500 hours and 24 surpassing 1k hours.<n>Fine-tuning existing ASR models on WorldSpeech results in an average relative Word-Error-Rate reduction of 63.5% across 11 typologically diverse languages.
  arXiv  Detail & Related papers  (2026-05-09T21:00:01Z)
• PashtoCorp: A 1.25-Billion-Word Corpus, Evaluation Suite, and Reproducible Pipeline for Low-Resource Language Development [0.0]
  PashtoCorp is a 1.25-billion-word corpus for Pashto, a language spoken by 60 million people that remains underrepresented in NLP.<n>At 1.25B words across 2.81 million documents, PashtoCorp is 40x larger than the OSCAR Pashto subset and 83x larger than the previously largest dedicated Pashto corpus.
  arXiv  Detail & Related papers  (2026-03-17T10:36:18Z)
• Quechua Speech Datasets in Common Voice: The Case of Puno Quechua [0.0]
  Under-resourced languages, such as Quechuas, face data and resource scarcity.<n>This paper examines the integration of Quechua languages into Common Voice.
  arXiv  Detail & Related papers  (2025-10-13T12:44:17Z)
• A Large Dataset of Spontaneous Speech with the Accent Spoken in São Paulo for Automatic Speech Recognition Evaluation [0.8929537856623965]
  We present a freely available spontaneous speech corpus for the Brazilian Portuguese language. We report preliminary automatic speech recognition (ASR) results, using both the Wav2Vec2-XLSR-53 and Distil-Whisper models. To the best of our knowledge, this is the first large Paulistano accented spontaneous speech corpus dedicated to the ASR task in Portuguese.
  arXiv  Detail & Related papers  (2024-09-10T21:45:06Z)
• ÌròyìnSpeech: A multi-purpose Yorùbá Speech Corpus [7.97238074132292]
  IroyinSpeech is a new corpus influenced by the desire to increase the amount of high quality, contemporary Yorub'a speech data. We curated about 23000 text sentences from news and creative writing domains with the open license CC-BY-4.0.
  arXiv  Detail & Related papers  (2023-07-29T20:42:50Z)
• ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation [79.66359274050885]
  We present ComSL, a speech-language model built atop a composite architecture of public pretrained speech-only and language-only models. Our approach has demonstrated effectiveness in end-to-end speech-to-text translation tasks.
  arXiv  Detail & Related papers  (2023-05-24T07:42:15Z)
• Scaling Speech Technology to 1,000+ Languages [66.31120979098483]
  The Massively Multilingual Speech (MMS) project increases the number of supported languages by 10-40x, depending on the task. Main ingredients are a new dataset based on readings of publicly available religious texts. We built pre-trained wav2vec 2.0 models covering 1,406 languages, a single multilingual automatic speech recognition model for 1,107 languages, speech synthesis models for the same number of languages, and a language identification model for 4,017 languages.
  arXiv  Detail & Related papers  (2023-05-22T22:09:41Z)
• XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale [48.0390317915984]
  XLS-R is a large-scale model for cross-lingual speech representation learning based on wav2vec 2.0. We train models with up to 2B parameters on nearly half a million hours of publicly available speech audio in 128 languages.
  arXiv  Detail & Related papers  (2021-11-17T18:49:42Z)
• GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio [88.20960848885575]
  GigaSpeech is a multi-domain English speech recognition corpus with 10,000 hours of high quality labeled audio suitable for supervised training. Around 40,000 hours of transcribed audio is first collected from audiobooks, podcasts and YouTube, covering both read and spontaneous speaking styles. For system training, GigaSpeech provides five subsets of different sizes, 10h, 250h, 1000h, 2500h, and 10000h.
  arXiv  Detail & Related papers  (2021-06-13T04:09:16Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)
• CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus [57.641761472372814]
  CoVoST is a multilingual speech-to-text translation corpus from 11 languages into English. It diversified with over 11,000 speakers and over 60 accents. CoVoST is released under CC0 license and free to use.
  arXiv  Detail & Related papers  (2020-02-04T14:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.