MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates

• URL: http://arxiv.org/abs/2603.23048v1
• Date: Tue, 24 Mar 2026 10:40:23 GMT
• Title: MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates
• Abstract summary: MSRHuBERT is a multi-sampling-rate adaptive pre-training method.<n>It maps raw waveforms from different sampling rates to a shared temporal resolution without resampling.<n>In experiments spanning 16 to 48 kHz, MSRHuBERT outperforms HuBERT on speech recognition and full-band speech reconstruction.
• Score: 54.185472273408266
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised learning (SSL) has advanced speech processing. However, existing speech SSL methods typically assume a single sampling rate and struggle with mixed-rate data due to temporal resolution mismatch. To address this limitation, we propose MSRHuBERT, a multi-sampling-rate adaptive pre-training method. Building on HuBERT, we replace its single-rate downsampling CNN with a multi-sampling-rate adaptive downsampling CNN that maps raw waveforms from different sampling rates to a shared temporal resolution without resampling. This design enables unified mixed-rate pre-training and fine-tuning. In experiments spanning 16 to 48 kHz, MSRHuBERT outperforms HuBERT on speech recognition and full-band speech reconstruction, preserving high-frequency detail while modeling low-frequency semantic structure. Moreover, MSRHuBERT retains HuBERT's mask-prediction objective and Transformer encoder, so existing analyses and improvements that were developed for HuBERT can apply directly.

Related papers

• Test-time adaptation for speech enhancement with an autoregressive speech prior [40.592330125629644]
  Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions.<n>We propose a single-utterance TTA method that regularizes a pretrained speech enhancement model using an autoregressive prior trained on clean speech latent representations extracted from a neural audio.
  arXiv  Detail & Related papers  (2026-09-03T10:10:23Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Acceleration [19.18455910385295]
  Token-Adaptive Predictor (TAP) is a training-free, probe-driven framework that adaptively selects a predictor for each token at every sampling step.<n>TAP incurs negligible overhead while enabling large speedups with little or no perceptual quality loss.
  arXiv  Detail & Related papers  (2026-03-04T07:10:11Z)
• SA-SSL-MOS: Self-supervised Learning MOS Prediction with Spectral Augmentation for Generalized Multi-Rate Speech Assessment [12.343358196209167]
  We propose a spectrogram-augmented SSL method that incorporates high-frequency features (up to 48 kHz sampling rate) through a parallel-branch architecture.<n> Experimental results show that leveraging high-frequency information overlooked by SSL features is crucial for accurate multi-rate SQA.
  arXiv  Detail & Related papers  (2026-02-16T14:33:56Z)
• FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling [59.8051705468084]
  Speculative sampling has emerged as an important technique for accelerating the auto-regressive generation process of large language models.<n>We present FR-Spec, a frequency-ranked speculative sampling framework that optimize draft candidate selection through vocabulary space compression.
  arXiv  Detail & Related papers  (2025-02-20T18:58:10Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation [21.335983674309475]
  Diffusion models suffer from slow inference due to an excessive number of queries to the underlying denoising network per generation. We introduce ConsistencyTTA, a framework requiring only a single non-autoregressive network query. We achieve so by proposing "CFG-aware latent consistency model," which adapts consistency generation into a latent space.
  arXiv  Detail & Related papers  (2023-09-19T16:36:33Z)
• HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition [11.243855639847514]
  We introduce an early exit scheme for ASR, namely HuBERT-EE, that allows the model to stop the inference dynamically. Experimental results on the LibriSpeech show that HuBERT-EE can accelerate the inference of the HuBERT while simultaneously balancing the trade-off between the performance and the latency.
  arXiv  Detail & Related papers  (2022-04-13T12:11:44Z)
• Neural Vocoder is All You Need for Speech Super-resolution [56.84715616516612]
  Speech super-resolution (SR) is a task to increase speech sampling rate by generating high-frequency components. Existing speech SR methods are trained in constrained experimental settings, such as a fixed upsampling ratio. We propose a neural vocoder based speech super-resolution method (NVSR) that can handle a variety of input resolution and upsampling ratios.
  arXiv  Detail & Related papers  (2022-03-28T17:51:00Z)
• MSR-NV: Neural Vocoder Using Multiple Sampling Rates [4.030676736893601]
  We propose a method to handle multiple sampling rates in a single neural vocoder (NV) By generating waveforms step-by-step starting from a low sampling rate, MSR-NV can efficiently learn the characteristics of each frequency band and synthesize high-quality speech at multiple sampling rates.
  arXiv  Detail & Related papers  (2021-09-28T13:31:20Z)
• Latent Template Induction with Gumbel-CRFs [107.17408593510372]
  We explore the use of structured variational autoencoders to infer latent templates for sentence generation. As a structured inference network, we show that it learns interpretable templates during training.
  arXiv  Detail & Related papers  (2020-11-29T01:00:57Z)
• Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation [66.46123655365113]
  Target speech separation refers to extracting the target speaker's speech from mixed signals. Two main challenges are the complex acoustic environment and the real-time processing requirement. We propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture.
  arXiv  Detail & Related papers  (2020-01-02T11:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.