Exploring Meta Information for Audio-based Zero-shot Bird Classification

• URL: http://arxiv.org/abs/2309.08398v2
• Date: Tue, 11 Jun 2024 17:29:51 GMT
• Title: Exploring Meta Information for Audio-based Zero-shot Bird Classification
• Authors: Alexander Gebhard, Andreas Triantafyllopoulos, Teresa Bez, Lukas Christ, Alexander Kathan, Björn W. Schuller,
• Abstract summary: This study investigates how meta-information can improve zero-shot audio classification. We use bird species as an example case study due to the availability of rich and diverse meta-data.
• Score: 113.17261694996051
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Advances in passive acoustic monitoring and machine learning have led to the procurement of vast datasets for computational bioacoustic research. Nevertheless, data scarcity is still an issue for rare and underrepresented species. This study investigates how meta-information can improve zero-shot audio classification, utilising bird species as an example case study due to the availability of rich and diverse meta-data. We investigate three different sources of metadata: textual bird sound descriptions encoded via (S)BERT, functional traits (AVONET), and bird life-history (BLH) characteristics. As audio features, we extract audio spectrogram transformer (AST) embeddings and project them to the dimension of the auxiliary information by adopting a single linear layer. Then, we employ the dot product as compatibility function and a standard zero-shot learning ranking hinge loss to determine the correct class. The best results are achieved by concatenating the AVONET and BLH features attaining a mean unweighted F1-score of .233 over five different test sets with 8 to 10 classes.

Related papers

• Reproducible Machine Learning-based Voice Pathology Detection: Introducing the Pitch Difference Feature [1.1455937444848385]
  We propose a robust set of features derived from a thorough research of contemporary practices in voice pathology detection. We combine this feature set, containing data from the publicly available Saarbr"ucken Voice Database (SVD), with preprocessing using the K-Means Synthetic Minority Over-Sampling Technique algorithm. Our approach has achieved the state-of-the-art performance, measured by unweighted average recall in voice pathology detection.
  arXiv  Detail & Related papers  (2024-10-14T14:17:52Z)
• Advanced Framework for Animal Sound Classification With Features Optimization [35.2832738406242]
  We propose an automated classification framework applicable to general animal sound classification. Our approach consistently outperforms baseline methods by over 25% in precision, recall, and accuracy.
  arXiv  Detail & Related papers  (2024-07-03T18:33:47Z)
• Text-to-feature diffusion for audio-visual few-shot learning [59.45164042078649]
  Few-shot learning from video data is a challenging and underexplored, yet much cheaper, setup. We introduce a unified audio-visual few-shot video classification benchmark on three datasets. We show that AV-DIFF obtains state-of-the-art performance on our proposed benchmark for audio-visual few-shot learning.
  arXiv  Detail & Related papers  (2023-09-07T17:30:36Z)
• Improving Primate Sounds Classification using Binary Presorting for Deep Learning [6.044912425856236]
  In this work, we introduce a generalized approach that first relabels subsegments of MEL spectrogram representations. For both the binary pre-sorting and the classification, we make use of convolutional neural networks (CNN) and various data-augmentation techniques. We showcase the results of this approach on the challenging textitComparE 2021 dataset, with the task of classifying between different primate species sounds.
  arXiv  Detail & Related papers  (2023-06-28T09:35:09Z)
• Feature-Rich Audio Model Inversion for Data-Free Knowledge Distillation Towards General Sound Classification [23.35582432472955]
  We propose feature-rich audio model inversion (FRAMI), a data-free knowledge distillation framework for general sound classification tasks. Experimental results on the Urbansound8k, ESC-50, and audioMNIST datasets demonstrate that FRAMI can generate feature-rich samples.
  arXiv  Detail & Related papers  (2023-03-14T06:04:19Z)
• Deep Feature Learning for Medical Acoustics [78.56998585396421]
  The purpose of this paper is to compare different learnables in medical acoustics tasks. A framework has been implemented to classify human respiratory sounds and heartbeats in two categories, i.e. healthy or affected by pathologies.
  arXiv  Detail & Related papers  (2022-08-05T10:39:37Z)
• Low-complexity deep learning frameworks for acoustic scene classification [64.22762153453175]
  We present low-complexity deep learning frameworks for acoustic scene classification (ASC) The proposed frameworks can be separated into four main steps: Front-end spectrogram extraction, online data augmentation, back-end classification, and late fusion of predicted probabilities. Our experiments conducted on DCASE 2022 Task 1 Development dataset have fullfiled the requirement of low-complexity and achieved the best classification accuracy of 60.1%.
  arXiv  Detail & Related papers  (2022-06-13T11:41:39Z)
• Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile [78.1212767880785]
  meta-learner is prone to overfitting since there are only a few available samples. When handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise. We present Eigen-Reptile (ER) that updates the meta- parameters with the main direction of historical task-specific parameters.
  arXiv  Detail & Related papers  (2022-06-04T08:48:02Z)
• Searching for Robustness: Loss Learning for Noisy Classification Tasks [81.70914107917551]
  We parameterize a flexible family of loss functions using Taylors and apply evolutionary strategies to search for noise-robust losses in this space. The resulting white-box loss provides a simple and fast "plug-and-play" module that enables effective noise-robust learning in diverse downstream tasks.
  arXiv  Detail & Related papers  (2021-02-27T15:27:22Z)
• An Ensemble of Convolutional Neural Networks for Audio Classification [9.174145063580882]
  ensembles of CNNs for audio classification are presented and tested on three freely available audio classification datasets. To the best of our knowledge, this is the most extensive study investigating ensembles of CNNs for audio classification.
  arXiv  Detail & Related papers  (2020-07-15T19:41:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.