Granary: Speech Recognition and Translation Dataset in 25 European Languages

• URL: http://arxiv.org/abs/2505.13404v2
• Date: Wed, 21 May 2025 17:00:54 GMT
• Title: Granary: Speech Recognition and Translation Dataset in 25 European Languages
• Authors: Nithin Rao Koluguri, Monica Sekoyan, George Zelenfroynd, Sasha Meister, Shuoyang Ding, Sofia Kostandian, He Huang, Nikolay Karpov, Jagadeesh Balam, Vitaly Lavrukhin, Yifan Peng, Sara Papi, Marco Gaido, Alessio Brutti, Boris Ginsburg,
• Abstract summary: Granary is a large-scale collection of speech datasets for recognition and translation across 25 European languages.<n>This is the first open-source effort at this scale for both transcription and translation.
• Score: 37.561934855489504
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-task and multilingual approaches benefit large models, yet speech processing for low-resource languages remains underexplored due to data scarcity. To address this, we present Granary, a large-scale collection of speech datasets for recognition and translation across 25 European languages. This is the first open-source effort at this scale for both transcription and translation. We enhance data quality using a pseudo-labeling pipeline with segmentation, two-pass inference, hallucination filtering, and punctuation restoration. We further generate translation pairs from pseudo-labeled transcriptions using EuroLLM, followed by a data filtration pipeline. Designed for efficiency, our pipeline processes vast amount of data within hours. We assess models trained on processed data by comparing their performance on previously curated datasets for both high- and low-resource languages. Our findings show that these models achieve similar performance using approx. 50% less data. Dataset will be made available at https://hf.co/datasets/nvidia/Granary

Related papers

• Self-supervised learning of speech representations with Dutch archival data [8.504327926435158]
  We show how music, noise and speaker overlap affect SSL convergence and downstream fine-tuning performance.<n>We convert the noisy broadcast dataset into a qualitative dataset for pre-training, by using Whisper and WhisperX.<n>Finally, we achieve a state-of-the-art large wav2vec 2.0 model for the Dutch language, by a continuation of pre-training a wav2vec 2.0 XLS-R model checkpoint with our 55k hour archival dataset.
  arXiv  Detail & Related papers  (2025-07-06T22:11:22Z)
• Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models [52.22235443948351]
  High-quality multilingual training data is essential for effectively pretraining large language models (LLMs)<n>Here, we introduce JQL, a systematic approach that efficiently curates diverse and high-quality multilingual data at scale.<n>JQL distills LLMs' annotation capabilities into lightweight annotators based on pretrained multilingual embeddings.
  arXiv  Detail & Related papers  (2025-05-28T11:06:54Z)
• From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora [85.44082712798553]
  We introduce a large-scale, high-quality multi-way parallel corpus, TED2025, based on TED Talks.<n>This dataset spans 113 languages, with up to 50 languages aligned in parallel, ensuring extensive multilingual coverage.<n>Experiments show that models trained on multiway parallel data consistently outperform those trained on unaligned multilingual data.
  arXiv  Detail & Related papers  (2025-05-20T07:43:45Z)
• Beyond Translation: LLM-Based Data Generation for Multilingual Fact-Checking [2.321323878201932]
  MultiSynFact is the first large-scale multilingual fact-checking dataset containing 2.2M claim-source pairs.<n>Our dataset generation pipeline leverages Large Language Models (LLMs), integrating external knowledge from Wikipedia.<n>We open-source a user-friendly framework to facilitate further research in multilingual fact-checking and dataset generation.
  arXiv  Detail & Related papers  (2025-02-21T12:38:26Z)
• Deepfake audio as a data augmentation technique for training automatic speech to text transcription models [55.2480439325792]
  We propose a framework that approaches data augmentation based on deepfake audio. A dataset produced by Indians (in English) was selected, ensuring the presence of a single accent.
  arXiv  Detail & Related papers  (2023-09-22T11:33:03Z)
• T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification [50.675552118811]
  Cross-lingual text classification is typically built on large-scale, multilingual language models (LMs) pretrained on a variety of languages of interest. We propose revisiting the classic "translate-and-test" pipeline to neatly separate the translation and classification stages.
  arXiv  Detail & Related papers  (2023-06-08T07:33:22Z)
• Multilingual Multimodal Learning with Machine Translated Text [27.7207234512674]
  We investigate whether machine translating English multimodal data can be an effective proxy for the lack of readily available multilingual data. We propose two metrics for automatically removing such translations from the resulting datasets. In experiments on five tasks across 20 languages in the IGLUE benchmark, we show that translated data can provide a useful signal for multilingual multimodal learning.
  arXiv  Detail & Related papers  (2022-10-24T11:41:20Z)
• Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings [10.871587311621974]
  We experimentally demonstrate that high capacity multilingual language models applied in a zero-shot setting consistently outperform data-based cross-lingual transfer approaches. A detailed analysis of our results suggests that this might be due to important differences in language use. Our results also indicate that data-based cross-lingual transfer approaches remain a competitive option when high-capacity multilingual language models are not available.
  arXiv  Detail & Related papers  (2022-10-23T05:37:35Z)
• Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking [84.50302759362698]
  We enhance the transfer learning process by intermediate fine-tuning of pretrained multilingual models. We use parallel and conversational movie subtitles datasets to design cross-lingual intermediate tasks. We achieve impressive improvements (> 20% on goal accuracy) on the parallel MultiWoZ dataset and Multilingual WoZ dataset.
  arXiv  Detail & Related papers  (2021-09-28T11:22:38Z)
• Multilingual Neural Semantic Parsing for Low-Resourced Languages [1.6244541005112747]
  We introduce a new multilingual semantic parsing dataset in English, Italian and Japanese. We show that joint multilingual training with pretrained encoders substantially outperforms our baselines on the TOP dataset. We find that a semantic trained only on English data achieves a zero-shot performance of 44.9% exact-match accuracy on Italian sentences.
  arXiv  Detail & Related papers  (2021-06-07T09:53:02Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)
• Mixed-Lingual Pre-training for Cross-lingual Summarization [54.4823498438831]
  Cross-lingual Summarization aims at producing a summary in the target language for an article in the source language. We propose a solution based on mixed-lingual pre-training that leverages both cross-lingual tasks like translation and monolingual tasks like masked language models. Our model achieves an improvement of 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over state-of-the-art results.
  arXiv  Detail & Related papers  (2020-10-18T00:21:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.