The Thiomi Dataset: A Large-Scale Multimodal Corpus for Low-Resource African Languages

• URL: http://arxiv.org/abs/2603.29244v2
• Date: Sun, 05 Apr 2026 22:04:59 GMT
• Title: The Thiomi Dataset: A Large-Scale Multimodal Corpus for Low-Resource African Languages
• Abstract summary: We present a large-scale multimodal corpus spanning ten African languages across four language families: Swahili, Kikuyu, Kamba, Kimeru, Luo, Maasai, Kipsigis, Somali.<n>The dataset contains over 601,000 approved sentence-level text annotations and over 385,000 audio recordings.<n>We describe the collection platform, quality assurance, and baseline experiments, and discuss implications for African language technology infrastructure.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present the Thiomi Dataset, a large-scale multimodal corpus spanning ten African languages across four language families: Swahili, Kikuyu, Kamba, Kimeru, Luo, Maasai, Kipsigis, Somali (East Africa); Wolof (West Africa); and Fulani (West/Central Africa). The dataset contains over 601,000 approved sentence-level text annotations and over 385,000 audio recordings, collected through a dedicated community data collection platform involving over 100 contributors. To validate the dataset's utility, we train and evaluate ASR, MT, and TTS models, establishing baselines across all languages. Our best ASR system achieves 3.24% WER on Swahili (Common Voice), reducing prior academic SOTA from 8.3% to 3.24% (5.1 percentage point absolute, 61% relative reduction), and 4.3% WER on Somali. The dataset will be published on HuggingFace. We describe the collection platform, quality assurance workflows, and baseline experiments, and discuss implications for African language technology infrastructure.

Related papers

• WAXAL: A Large-Scale Multilingual African Language Speech Corpus [12.433885475371035]
  WAXAL is a large-scale, openly accessible speech dataset for 21 languages representing over 100 million speakers.<n>The collection consists of two main components: an Automated Speech Recognition (ASR) dataset containing approximately 1,250 hours of transcribed, natural speech from a diverse range of speakers, and a Text-to-Speech (TTS) dataset with over 180 hours of high-quality, single-speaker recordings reading phonetically balanced scripts.
  arXiv  Detail & Related papers  (2026-02-02T19:49:19Z)
• The African Languages Lab: A Collaborative Approach to Advancing Low-Resource African NLP [4.188487384419692]
  Despite representing nearly one-third of the world's languages, African languages remain critically underserved by modern NLP technologies.<n>We present the African Languages Lab, a comprehensive research initiative that addresses this technological gap through systematic data collection, model development, and capacity building.
  arXiv  Detail & Related papers  (2025-10-07T07:42:52Z)
• The NaijaVoices Dataset: Cultivating Large-Scale, High-Quality, Culturally-Rich Speech Data for African Languages [10.225163354933372]
  We introduce the NaijaVoices dataset, a 1,800-hour speech-text dataset with 5,000+ speakers.<n>We outline our unique data collection approach, analyze its acoustic diversity, and demonstrate its impact through finetuning experiments.<n>These results highlight NaijaVoices' potential to advance multilingual speech processing for African languages.
  arXiv  Detail & Related papers  (2025-05-26T22:53:48Z)
• Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments [0.9214083577876088]
  This paper creates approximately 1 million human-translated words of new benchmark data in 8 low-resource African languages.<n>Our benchmarks are translations of Winogrande and three sections of MMLU: college medicine, clinical knowledge, and virology.<n>Using the benchmarks translated, we report previously unknown performance gaps between state-of-the-art (SOTA) LLMs in English and African languages.
  arXiv  Detail & Related papers  (2024-12-16T23:50:21Z)
• Voices Unheard: NLP Resources and Models for Yorùbá Regional Dialects [72.18753241750964]
  Yorub'a is an African language with roughly 47 million speakers. Recent efforts to develop NLP technologies for African languages have focused on their standard dialects. We take steps towards bridging this gap by introducing a new high-quality parallel text and speech corpus.
  arXiv  Detail & Related papers  (2024-06-27T22:38:04Z)
• AfroBench: How Good are Large Language Models on African Languages? [55.35674466745322]
  AfroBench is a benchmark for evaluating the performance of LLMs across 64 African languages.<n>AfroBench consists of nine natural language understanding datasets, six text generation datasets, six knowledge and question answering tasks, and one mathematical reasoning task.
  arXiv  Detail & Related papers  (2023-11-14T08:10:14Z)
• AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages [45.88640066767242]
  Africa is home to over 2,000 languages from more than six language families and has the highest linguistic diversity among all continents. Yet, there is little NLP research conducted on African languages. Crucial to enabling such research is the availability of high-quality annotated datasets. In this paper, we introduce AfriSenti, a sentiment analysis benchmark that contains a total of >110,000 tweets in 14 African languages.
  arXiv  Detail & Related papers  (2023-02-17T15:40:12Z)
• MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition [55.95128479289923]
  African languages are spoken by over a billion people, but are underrepresented in NLP research and development. We create the largest human-annotated NER dataset for 20 African languages. We show that choosing the best transfer language improves zero-shot F1 scores by an average of 14 points.
  arXiv  Detail & Related papers  (2022-10-22T08:53:14Z)
• Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages [62.730361829175415]
  MIRACL is a multilingual dataset we have built for the WSDM 2023 Cup challenge. It focuses on ad hoc retrieval across 18 different languages. Our goal is to spur research that will improve retrieval across a continuum of languages.
  arXiv  Detail & Related papers  (2022-10-18T16:47:18Z)
• ASR2K: Speech Recognition for Around 2000 Languages without Audio [100.41158814934802]
  We present a speech recognition pipeline that does not require any audio for the target language. Our pipeline consists of three components: acoustic, pronunciation, and language models. We build speech recognition for 1909 languages by combining it with Crubadan: a large endangered languages n-gram database.
  arXiv  Detail & Related papers  (2022-09-06T22:48:29Z)
• CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus [57.641761472372814]
  CoVoST is a multilingual speech-to-text translation corpus from 11 languages into English. It diversified with over 11,000 speakers and over 60 accents. CoVoST is released under CC0 license and free to use.
  arXiv  Detail & Related papers  (2020-02-04T14:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.