Oral to Web: Digitizing 'Zero Resource'Languages of Bangladesh

• URL: http://arxiv.org/abs/2603.05272v1
• Date: Thu, 05 Mar 2026 15:20:57 GMT
• Title: Oral to Web: Digitizing 'Zero Resource'Languages of Bangladesh
• Abstract summary: The Multilingual Cloud Corpus is the first national-scale, parallel, multimodal linguistic dataset of Bangladesh's ethnic and indigenous languages.<n>It comprises 85792 structured textual entries, each containing a Bengali stimulus text, an English translation, and an IPA transcription, together with 107 hours of transcribed audio recordings.<n>The data were collected through systematic fieldwork over 90 days across nine districts of Bangladesh, involving 16 data collectors, 77 speakers, and 43 validators.
• Score: 1.0578042460060062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the Multilingual Cloud Corpus, the first national-scale, parallel, multimodal linguistic dataset of Bangladesh's ethnic and indigenous languages. Despite being home to approximately 40 minority languages spanning four language families, Bangladesh has lacked a systematic, cross-family digital corpus for these predominantly oral, computationally "zero resource" varieties, 14 of which are classified as endangered. Our corpus comprises 85792 structured textual entries, each containing a Bengali stimulus text, an English translation, and an IPA transcription, together with approximately 107 hours of transcribed audio recordings, covering 42 language varieties from the Tibeto-Burman, Indo-European, Austro-Asiatic, and Dravidian families, plus two genetically unclassified languages. The data were collected through systematic fieldwork over 90 days across nine districts of Bangladesh, involving 16 data collectors, 77 speakers, and 43 validators, following a predefined elicitation template of 2224 unique items organized at three levels of linguistic granularity: isolated lexical items (475 words across 22 semantic domains), grammatical constructions (887 sentences across 21 categories including verbal conjugation paradigms), and directed speech (862 prompts across 46 conversational scenarios). Post-field processing included IPA transcription by 10 linguists with independent adjudication by 6 reviewers. The complete dataset is publicly accessible through the Multilingual Cloud platform (multiling.cloud), providing searchable access to annotated audio and textual data for all documented varieties. We describe the corpus design, fieldwork methodology, dataset structure, and per-language coverage, and discuss implications for endangered language documentation, low-resource NLP, and digital preservation in linguistically diverse developing countries.

Related papers

• Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking [72.66384771245389]
  This work introduces HEALTHDIAL, a large-scale, multilingual, and multi-parallel dataset for developing and evaluating retrieval-augmented generation (RAG)-based spoken dialogue systems.<n>The dataset comprises 6,000 information-seeking dialogues (1,500 per language) grounded in trusted content from the World Health Organization (WHO) and 163 hours of user speech recorded from native speakers of diverse dialects.
  arXiv  Detail & Related papers  (2026-05-28T15:47:09Z)
• CS-FLEURS: A Massively Multilingual and Code-Switched Speech Dataset [99.0507412649934]
  CS-FLEURS consists of 4 test sets which cover in total 113 unique code-switched language pairs across 52 languages.<n> CS-FLEURS also provides a training set with 128 hours of generative text-to-speech data across 16 X-English language pairs.
  arXiv  Detail & Related papers  (2025-09-17T16:45:22Z)
• Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languages [27.273651323572786]
  BhasaAnuvaad is the largest speech translation dataset for Indian languages, spanning over 44 thousand hours of audio and 17 million aligned text segments.<n>Our experiments demonstrate improvements in the translation quality, setting a new standard for Indian language speech translation.<n>We will release all the code, data and model weights in the open-source, with permissive licenses to promote accessibility and collaboration.
  arXiv  Detail & Related papers  (2024-11-07T13:33:34Z)
• Wav2Gloss: Generating Interlinear Glossed Text from Speech [78.64412090339044]
  We propose Wav2Gloss, a task in which four linguistic annotation components are extracted automatically from speech. We provide various baselines to lay the groundwork for future research on Interlinear Glossed Text generation from speech.
  arXiv  Detail & Related papers  (2024-03-19T21:45:29Z)
• IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages [17.862027695142825]
  INDICVOICES is a dataset of natural and spontaneous speech from 16237 speakers covering 145 Indian districts and 22 languages. 1639 hours have already been transcribed, with a median of 73 hours per language. All the data, tools, guidelines, models and other materials developed as a part of this work will be made publicly available.
  arXiv  Detail & Related papers  (2024-03-04T10:42:08Z)
• NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages [54.808217147579036]
  We conduct a case study on Indonesian local languages. We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets. Our findings demonstrate that datasets generated through paragraph writing by native speakers exhibit superior quality in terms of lexical diversity and cultural content.
  arXiv  Detail & Related papers  (2023-09-19T14:42:33Z)
• IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages [37.758476568195256]
  India has a rich linguistic landscape with languages from 4 major language families spoken by over a billion people. 22 of these languages are listed in the Constitution of India (referred to as scheduled languages)
  arXiv  Detail & Related papers  (2023-05-25T17:57:43Z)
• UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languages [0.0]
  We show how a multilingual large language model can be a resource for sentiment analysis in languages not seen during pretraining. The languages are to various degrees related to languages used during pretraining, and the language data contain various degrees of code-switching. We experiment with both monolingual and multilingual datasets for the final fine-tuning, and find that with the provided datasets that contain samples in the thousands, monolingual fine-tuning yields the best results.
  arXiv  Detail & Related papers  (2023-04-27T13:51:18Z)
• Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages [62.730361829175415]
  MIRACL is a multilingual dataset we have built for the WSDM 2023 Cup challenge. It focuses on ad hoc retrieval across 18 different languages. Our goal is to spur research that will improve retrieval across a continuum of languages.
  arXiv  Detail & Related papers  (2022-10-18T16:47:18Z)
• Annotated Speech Corpus for Low Resource Indian Languages: Awadhi, Bhojpuri, Braj and Magahi [2.84214511742034]
  We develop a speech corpus for four low-resource Indo-Aryan languages -- Awadhi, Bhojpuri, Braj and Magahi. The total size of the corpus currently stands at approximately 18 hours. We discuss our methodology for data collection in these languages, most of which was done in the middle of the COVID-19 pandemic.
  arXiv  Detail & Related papers  (2022-06-26T17:28:38Z)
• SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection [81.85463892070085]
  The SIGMORPHON 2020 task on morphological reinflection aims to investigate systems' ability to generalize across typologically distinct languages. Systems were developed using data from 45 languages and just 5 language families, fine-tuned with data from an additional 45 languages and 10 language families (13 in total), and evaluated on all 90 languages.
  arXiv  Detail & Related papers  (2020-06-20T13:24:14Z)
• CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus [57.641761472372814]
  CoVoST is a multilingual speech-to-text translation corpus from 11 languages into English. It diversified with over 11,000 speakers and over 60 accents. CoVoST is released under CC0 license and free to use.
  arXiv  Detail & Related papers  (2020-02-04T14:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.