How Class Ontology and Data Scale Affect Audio Transfer Learning

• URL: http://arxiv.org/abs/2603.25476v1
• Date: Thu, 26 Mar 2026 14:18:29 GMT
• Title: How Class Ontology and Data Scale Affect Audio Transfer Learning
• Abstract summary: Transfer learning is a crucial concept within deep learning that allows artificial neural networks to benefit from a large pre-training data basis.<n>We study audio-to-audio transfer learning, in which we pre-train various model states on subsets of AudioSet.<n>We report that increasing the number of samples and classes in the pre-training data both have a positive impact on transfer learning.
• Score: 86.30979869026778
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Transfer learning is a crucial concept within deep learning that allows artificial neural networks to benefit from a large pre-training data basis when confronted with a task of limited data. Despite its ubiquitous use and clear benefits, there are still many open questions regarding the inner workings of transfer learning and, in particular, regarding the understanding of when and how well it works. To that extent, we perform a rigorous study focusing on audio-to-audio transfer learning, in which we pre-train various model states on (ontology-based) subsets of AudioSet and fine-tune them on three computer audition tasks, namely acoustic scene recognition, bird activity recognition, and speech command recognition. We report that increasing the number of samples and classes in the pre-training data both have a positive impact on transfer learning. This is, however, generally surpassed by similarity between pre-training and the downstream task, which can lead the model to learn comparable features.

Related papers

• Biomimetic Frontend for Differentiable Audio Processing [5.13105998753022]
  We build on a classical model of human hearing and make it differentiable, so that we can combine traditional explainable signal processing approaches with deep-learning frameworks. This allows us to arrive at an expressive and explainable model that is easily trained on modest amounts of data. Results show that our differentiable model surpasses black-box approaches in terms of computational efficiency and robustness, even with little training data.
  arXiv  Detail & Related papers  (2024-09-13T17:23:42Z)
• Sequential Contrastive Audio-Visual Learning [12.848371604063168]
  We propose sequential contrastive audiovisual learning (SCAV), which contrasts examples based on their non-aggregated representation space.<n>Experiments with the VGGSound and Music datasets demonstrate the effectiveness of SCAV.<n>We also show that models trained with SCAV exhibit a significant degree of flexibility regarding the metric employed for retrieval.
  arXiv  Detail & Related papers  (2024-07-08T09:45:20Z)
• ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification [42.95038619688867]
  ASiT is a novel self-supervised learning framework that captures local and global contextual information by employing group masked model learning and self-distillation. We evaluate our pretrained models on both audio and speech classification tasks, including audio event classification, keyword spotting, and speaker identification.
  arXiv  Detail & Related papers  (2022-11-23T18:21:09Z)
• Deep Feature Learning for Medical Acoustics [78.56998585396421]
  The purpose of this paper is to compare different learnables in medical acoustics tasks. A framework has been implemented to classify human respiratory sounds and heartbeats in two categories, i.e. healthy or affected by pathologies.
  arXiv  Detail & Related papers  (2022-08-05T10:39:37Z)
• Self-Supervised Learning for speech recognition with Intermediate layer supervision [52.93758711230248]
  We propose Intermediate Layer Supervision for Self-Supervised Learning (ILS-SSL) ILS-SSL forces the model to concentrate on content information as much as possible by adding an additional SSL loss on the intermediate layers. Experiments on LibriSpeech test-other set show that our method outperforms HuBERT significantly.
  arXiv  Detail & Related papers  (2021-12-16T10:45:05Z)
• Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance [55.10864476206503]
  We investigate the use of quantized vectors to model the latent linguistic embedding. By enforcing different policies over the latent spaces in the training, we are able to obtain a latent linguistic embedding. Our experiments show that the voice cloning system built with vector quantization has only a small degradation in terms of perceptive evaluations.
  arXiv  Detail & Related papers  (2021-06-25T07:51:35Z)
• Do sound event representations generalize to other audio tasks? A case study in audio transfer learning [20.572846660950812]
  In this paper, we investigate transfer learning capacity of audio representations obtained from neural networks trained on a large-scale sound event detection dataset. We show that such simple linear transfer is already powerful enough to achieve high performance on the downstream tasks.
  arXiv  Detail & Related papers  (2021-06-21T18:04:59Z)
• Robust Audio-Visual Instance Discrimination [79.74625434659443]
  We present a self-supervised learning method to learn audio and video representations. We address the problems of audio-visual instance discrimination and improve transfer learning performance.
  arXiv  Detail & Related papers  (2021-03-29T19:52:29Z)
• What is being transferred in transfer learning? [51.6991244438545]
  We show that when training from pre-trained weights, the model stays in the same basin in the loss landscape. We present that when training from pre-trained weights, the model stays in the same basin in the loss landscape and different instances of such model are similar in feature space and close in parameter space.
  arXiv  Detail & Related papers  (2020-08-26T17:23:40Z)
• A Transfer Learning Method for Speech Emotion Recognition from Automatic Speech Recognition [0.0]
  We show a transfer learning method in speech emotion recognition based on a Time-Delay Neural Network architecture. We achieve the highest significantly higher accuracy when compared to state-of-the-art, using five-fold cross validation.
  arXiv  Detail & Related papers  (2020-08-06T20:37:22Z)
• A Sequential Self Teaching Approach for Improving Generalization in Sound Event Recognition [11.559570255513217]
  We propose a sequential self-teaching approach to learning sounds. It is harder to learn sounds in adverse situations such as from weakly labeled and/or noisy labeled data. Our proposal is a sequential stage-wise learning process that improves generalization capabilities of a given modeling system.
  arXiv  Detail & Related papers  (2020-06-30T22:53:43Z)
• Inter- and Intra-domain Knowledge Transfer for Related Tasks in Deep Character Recognition [2.320417845168326]
  Pre-training a deep neural network on the ImageNet dataset is a common practice for training deep learning models. The technique of pre-training on one task and then retraining on a new one is called transfer learning. In this paper we analyse the effectiveness of using deep transfer learning for character recognition tasks.
  arXiv  Detail & Related papers  (2020-01-02T14:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.