SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation

• URL: http://arxiv.org/abs/2512.21204v1
• Date: Wed, 24 Dec 2025 14:33:16 GMT
• Title: SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation
• Authors: Mahi Luthra, Jiayi Shen, Maxime Poli, Angelo Ortiz, Yosuke Higuchi, Youssef Benchekroun, Martin Gleize, Charles-Eric Saint-James, Dongyan Lin, Phillip Rust, Angel Villar, Surya Parimi, Vanessa Stark, Rashel Moritz, Juan Pino, Yann LeCun, Emmanuel Dupoux,
• Abstract summary: We introduce SpidR-Adapt for rapid adaptation to new languages using minimal unlabeled data.<n>We construct a multi-task adaptive pre-training protocol which formulates the adaptation process as a bi-level optimization framework.<n> Empirically, SpidR-Adapt achieves rapid gains in phonemic discriminability and spoken language modeling.
• Score: 40.55805997909858
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Human infants, with only a few hundred hours of speech exposure, acquire basic units of new languages, highlighting a striking efficiency gap compared to the data-hungry self-supervised speech models. To address this gap, this paper introduces SpidR-Adapt for rapid adaptation to new languages using minimal unlabeled data. We cast such low-resource speech representation learning as a meta-learning problem and construct a multi-task adaptive pre-training (MAdaPT) protocol which formulates the adaptation process as a bi-level optimization framework. To enable scalable meta-training under this framework, we propose a novel heuristic solution, first-order bi-level optimization (FOBLO), avoiding heavy computation costs. Finally, we stabilize meta-training by using a robust initialization through interleaved supervision which alternates self-supervised and supervised objectives. Empirically, SpidR-Adapt achieves rapid gains in phonemic discriminability (ABX) and spoken language modeling (sWUGGY, sBLIMP, tSC), improving over in-domain language models after training on less than 1h of target-language audio, over $100\times$ more data-efficient than standard training. These findings highlight a practical, architecture-agnostic path toward biologically inspired, data-efficient representations. We open-source the training code and model checkpoints at https://github.com/facebookresearch/spidr-adapt.

Related papers

• Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages [11.808922632545874]
  We analyze multilingual automatic speech recognition models and reveal a U-shaped adaptability pattern.<n>We propose DAMA, a Depth-Aware Model Adaptation framework that allocates adaptation capacity according to each layer's role.<n>Dama matches or surpasses state-of-the-art accuracy with 80% fewer trainable parameters, achieves a 29% error reduction under extreme data scarcity, and significantly improves memory, training time, and computational efficiency over baselines.
  arXiv  Detail & Related papers  (2026-02-01T04:18:31Z)
• Harmonizing the Arabic Audio Space with Data Scheduling [15.84874997729878]
  This paper presents the first systematic study of multi-task instruction tuning for an Arabic-centric audio LLM.<n>We fine-tune Qwen2.5- Omni (7B) and propose Task-Progressive Curriculum (TPC) along with Aligner-Based Diverse Sampling (ADS)<n>Our results reveal a critical efficiency, robustness trade-off: while ADS accelerates initial convergence, its inherent gradient volatility can destabilize generative decoding under prolonged training.
  arXiv  Detail & Related papers  (2026-01-18T17:08:31Z)
• SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervision [25.71776883846138]
  SpidR is a self-supervised speech representation model that efficiently learns representations with highly accessible phonetic information.<n>It is trained on raw waveforms using a masked prediction objective combined with self-distillation and online clustering.<n>It outperforms wav2vec 2.0, HuBERT, WavLM, and DinoSR on downstream language modeling benchmarks.
  arXiv  Detail & Related papers  (2025-12-23T12:22:25Z)
• How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario [72.02391485962127]
  Speech Self-Supervised Learning (SSL) models achieve impressive performance on Automatic Speech Recognition (ASR)<n>In low-resource language ASR, they encounter the domain mismatch problem between pre-trained and low-resource languages.<n>We extend a conventional efficient fine-tuning scheme based on the adapter to handle these issues.
  arXiv  Detail & Related papers  (2024-11-27T10:51:00Z)
• Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Model [57.78191634042409]
  We propose Pseudo-Word HuBERT (PW-HuBERT), a framework that integrates pseudo word-level targets into the training process. Our experimental results on four spoken language understanding (SLU) benchmarks suggest the superiority of our model in capturing semantic information.
  arXiv  Detail & Related papers  (2024-02-08T16:55:21Z)
• Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification [19.893213508284813]
  Self-supervised adaptive pre-training is proposed to adapt the pre-trained model to the target domain and languages of the downstream task. We show that SAPT improves XLSR performance on the FLEURS benchmark with substantial gains up to 40.1% for under-represented languages.
  arXiv  Detail & Related papers  (2023-12-12T14:58:08Z)
• Reduce, Reuse, Recycle: Is Perturbed Data better than Other Language augmentation for Low Resource Self-Supervised Speech Models [48.44820587495038]
  Self-supervised representation learning (SSRL) has demonstrated superior performance than supervised models for tasks including phoneme recognition. Training SSRL models poses a challenge for low-resource languages where sufficient pre-training data may not be available. We propose to use audio augmentation techniques, namely: pitch variation, noise addition, accented target language and other language speech to pre-train SSRL models in a low resource condition and evaluate phoneme recognition.
  arXiv  Detail & Related papers  (2023-09-22T10:09:09Z)
• eP-ALM: Efficient Perceptual Augmentation of Language Models [70.47962271121389]
  We propose to direct effort to efficient adaptations of existing models, and propose to augment Language Models with perception. Existing approaches for adapting pretrained models for vision-language tasks still rely on several key components that hinder their efficiency. We show that by freezing more than 99% of total parameters, training only one linear projection layer, and prepending only one trainable token, our approach (dubbed eP-ALM) significantly outperforms other baselines on VQA and Captioning.
  arXiv  Detail & Related papers  (2023-03-20T19:20:34Z)
• Learning Large-scale Neural Fields via Context Pruned Meta-Learning [60.93679437452872]
  We introduce an efficient optimization-based meta-learning technique for large-scale neural field training. We show how gradient re-scaling at meta-test time allows the learning of extremely high-quality neural fields. Our framework is model-agnostic, intuitive, straightforward to implement, and shows significant reconstruction improvements for a wide range of signals.
  arXiv  Detail & Related papers  (2023-02-01T17:32:16Z)
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition [30.460501537763736]
  We introduce an adaptive activation network to the upper layers of ASR model. We also proposed two approaches to train the model: (1) cross-lingual learning, replacing the activation function from source language to target language, and (2) multilingual learning. Our experiments on IARPA Babel datasets demonstrated that our approaches outperform the from-scratch training and traditional bottleneck feature based methods.
  arXiv  Detail & Related papers  (2022-05-28T04:02:59Z)
• From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension [51.953428342923885]
  We develop a two-stage approach to enhance the model performance. The first stage targets at recall: we design a hard-learning (HL) algorithm to maximize the likelihood that the top-k predictions contain the accurate answer. The second stage focuses on precision: an answer-aware contrastive learning mechanism is developed to learn the fine difference between the accurate answer and other candidates.
  arXiv  Detail & Related papers  (2021-12-09T07:31:15Z)
• From Universal Language Model to Downstream Task: Improving RoBERTa-Based Vietnamese Hate Speech Detection [8.602181445598776]
  We propose a pipeline to adapt the general-purpose RoBERTa language model to a specific text classification task: Vietnamese Hate Speech Detection. Our experiments proved that our proposed pipeline boosts the performance significantly, achieving a new state-of-the-art on Vietnamese Hate Speech Detection campaign with 0.7221 F1 score.
  arXiv  Detail & Related papers  (2021-02-24T09:30:55Z)
• Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling [30.905849959257264]
  This study addresses unsupervised subword modeling, i.e., learning feature representations that can distinguish subword units of a language. The proposed approach adopts a two-stage bottleneck feature (BNF) learning framework, consisting of autoregressive predictive coding ( APC) as a front-end and a DNN-BNF model as a back-end. The results on Libri-light and the ZeroSpeech 2017 databases show that APC is effective in front-end feature pretraining.
  arXiv  Detail & Related papers  (2020-07-25T19:41:41Z)
• Pre-training Text Representations as Meta Learning [113.3361289756749]
  We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks. We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
  arXiv  Detail & Related papers  (2020-04-12T09:05:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.