Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond

• URL: http://arxiv.org/abs/2408.03900v1
• Date: Wed, 7 Aug 2024 16:55:28 GMT
• Title: Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond
• Authors: Beomseok Lee, Ioan Calapodescu, Marco Gaido, Matteo Negri, Laurent Besacier,
• Abstract summary: Speech-MASSIVE is a multilingual Spoken Language Understanding dataset. It covers 12 languages from different families and inherits from the annotations for the intent prediction and slot-filling tasks. We demonstrate the suitability of Speech-MASSIVE for other tasks such as speech transcription, language identification, and speech translation.
• Score: 36.660499609887886
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present Speech-MASSIVE, a multilingual Spoken Language Understanding (SLU) dataset comprising the speech counterpart for a portion of the MASSIVE textual corpus. Speech-MASSIVE covers 12 languages from different families and inherits from MASSIVE the annotations for the intent prediction and slot-filling tasks. Our extension is prompted by the scarcity of massively multilingual SLU datasets and the growing need for versatile speech datasets to assess foundation models (LLMs, speech encoders) across languages and tasks. We provide a multimodal, multitask, multilingual dataset and report SLU baselines using both cascaded and end-to-end architectures in various training scenarios (zero-shot, few-shot, and full fine-tune). Furthermore, we demonstrate the suitability of Speech-MASSIVE for benchmarking other tasks such as speech transcription, language identification, and speech translation. The dataset, models, and code are publicly available at: https://github.com/hlt-mt/Speech-MASSIVE

Related papers

• MSLM-S2ST: A Multitask Speech Language Model for Textless Speech-to-Speech Translation with Speaker Style Preservation [45.558316325252335]
  Multitask Speech Language Model (MSLM) is a decoder-only speech language model trained in a multitask setting. Our model is able to support multilingual S2ST with speaker style preserved.
  arXiv  Detail & Related papers  (2024-03-19T03:35:20Z)
• Toward Joint Language Modeling for Speech Units and Text [89.32163954508489]
  We explore joint language modeling for speech units and text. We introduce automatic metrics to evaluate how well the joint LM mixes speech and text. Our results show that by mixing speech units and text with our proposed mixing techniques, the joint LM improves over a speech-only baseline on SLU tasks.
  arXiv  Detail & Related papers  (2023-10-12T20:53:39Z)
• Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation [65.13824257448564]
  This paper proposes a textless training method for many-to-many multilingual speech-to-speech translation. By treating the speech units as pseudo-text, we can focus on the linguistic content of the speech. We demonstrate that the proposed UTUT model can be effectively utilized not only for Speech-to-Speech Translation (S2ST) but also for multilingual Text-to-Speech Synthesis (T2S) and Text-to-Speech Translation (T2ST)
  arXiv  Detail & Related papers  (2023-08-03T15:47:04Z)
• AudioPaLM: A Large Language Model That Can Speak and Listen [79.44757696533709]
  We introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models. It can process and generate text and speech with applications including speech recognition and speech-to-speech translation.
  arXiv  Detail & Related papers  (2023-06-22T14:37:54Z)
• The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translation [13.352795145385645]
  Speech translation (ST) is a good means of pretraining speech models for end-to-end spoken language understanding. We show that our models reach higher performance over baselines on monolingual and multilingual intent classification. We also create new benchmark datasets for speech summarization and low-resource/zero-shot transfer from English to French or Spanish.
  arXiv  Detail & Related papers  (2023-05-16T17:53:03Z)
• SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks [88.4408774253634]
  Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community. There are not nearly as many SLU task benchmarks, and many of the existing ones use data that is not freely available to all researchers. Recent work has begun to introduce such benchmark for several tasks.
  arXiv  Detail & Related papers  (2022-12-20T18:39:59Z)
• SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation [11.552745999302905]
  We propose the SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation learning framework. We combine state-of-the-art multilingual acoustic frame-level speech representation learning model XLS-R with the Language Agnostic BERT Sentence Embedding (LaBSE) model to create an utterance-level multimodal multilingual speech encoder SAMU-XLSR.
  arXiv  Detail & Related papers  (2022-05-17T08:58:48Z)
• XTREME-S: Evaluating Cross-lingual Speech Representations [88.78720838743772]
  XTREME-S is a new benchmark to evaluate universal cross-lingual speech representations in many languages. This paper describes the new benchmark and establishes the first speech-only and speech-text baselines.
  arXiv  Detail & Related papers  (2022-03-21T06:50:21Z)
• Multilingual Speech Translation with Unified Transformer: Huawei Noah's Ark Lab at IWSLT 2021 [33.876412404781846]
  This paper describes the system submitted to the IWSLT 2021 Speech Translation (MultiST) task from Huawei Noah's Ark Lab. We use a unified transformer architecture for our MultiST model, so that the data from different modalities can be exploited to enhance the model's ability. We apply several training techniques to improve the performance, including multi-task learning, task-level curriculum learning, data augmentation, etc.
  arXiv  Detail & Related papers  (2021-06-01T02:50:49Z)
• MLS: A Large-Scale Multilingual Dataset for Speech Research [37.803100082550294]
  The dataset is derived from read audiobooks from LibriVox. It consists of 8 languages, including about 44.5K hours of English and a total of about 6K hours for other languages.
  arXiv  Detail & Related papers  (2020-12-07T01:53:45Z)
• SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding [61.02342238771685]
  Spoken language understanding requires a model to analyze input acoustic signal to understand its linguistic content and make predictions. Various pre-training methods have been proposed to learn rich representations from large-scale unannotated speech and text. We propose a novel semi-supervised learning framework, SPLAT, to jointly pre-train the speech and language modules.
  arXiv  Detail & Related papers  (2020-10-05T19:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.