Can Masked Autoencoders Also Listen to Birds?

• URL: http://arxiv.org/abs/2504.12880v3
• Date: Fri, 06 Jun 2025 09:48:16 GMT
• Title: Can Masked Autoencoders Also Listen to Birds?
• Authors: Lukas Rauch, René Heinrich, Ilyass Moummad, Alexis Joly, Bernhard Sick, Christoph Scholz,
• Abstract summary: Masked Autoencoders (MAEs) have shown competitive results in audio classification by learning rich semantic representations.<n>General-purpose models fail to generalize well when applied directly to fine-grained audio domains.<n>This work demonstrates that bridging this domain gap requires more than domain-specific pretraining data.
• Score: 2.430300340530418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Masked Autoencoders (MAEs) have shown competitive results in audio classification by learning rich semantic representations through an efficient self-supervised reconstruction task. However, general-purpose models fail to generalize well when applied directly to fine-grained audio domains. Specifically, bird-sound classification requires distinguishing subtle inter-species differences and managing high intra-species acoustic variability, thereby revealing the performance limitations of general-domain Audio-MAE models. This work demonstrates that bridging this domain gap requires more than domain-specific pretraining data; adapting the entire training pipeline is crucial. We systematically revisit and adapt the pretraining recipe, fine-tuning methods, and frozen feature utilization to bird sounds using BirdSet, a large-scale bioacoustic dataset comparable to AudioSet. Our resulting Bird-MAE achieves new state-of-the-art results in BirdSet's multi-label classification benchmark. Additionally, we introduce the parameter-efficient prototypical probing, enhancing the utility of frozen MAE representations and closely approaching fine-tuning performance in low-resource settings. Bird-MAE's prototypical probes outperform linear probing by up to 37%$_\text{p}$ in MAP and narrow the gap to fine-tuning to approximately 3.3%$_\text{p}$ on average across BirdSet downstream tasks. Bird-MAE also demonstrates robust few-shot capabilities with prototypical probing in our newly established few-shot benchmark on BirdSet, highlighting the potential of tailored self-supervised learning pipelines for fine-grained audio domains.

Related papers

• Foundation Models for Bioacoustics -- a Comparative Review [0.9109149174920012]
  We review bioacoustic foundation models by thoroughly analysing design decisions such as model architecture, pretraining scheme, and training paradigm.<n>We evaluate selected foundation models on classification tasks from the BEANS and BirdSet benchmarks.<n>Our comprehensive experimental analysis reveals that BirdMAE, trained on large-scale bird song data with a self-supervised objective, achieves the best performance on the BirdSet benchmark.
  arXiv  Detail & Related papers  (2025-08-02T09:15:16Z)
• An Automated Pipeline for Few-Shot Bird Call Classification: A Case Study with the Tooth-Billed Pigeon [0.6282171844772422]
  This paper presents an automated one-shot bird call classification pipeline designed for rare species absent from large publicly available classifiers like BirdNET and Perch. We leverage the embedding space of large bird classification networks and develop a classifier using cosine similarity, combined with filtering and denoising preprocessing techniques. The final model achieved 1.0 recall and 0.95 accuracy in detecting tooth-billed pigeon calls, making it practical for use in the field.
  arXiv  Detail & Related papers  (2025-04-22T21:21:41Z)
• A Bird Song Detector for improving bird identification through Deep Learning: a case study from Doñana [2.7924253850013416]
  We develop a pipeline for automatic bird vocalization identification in Donana National Park (SW Spain)<n>We manually annotated 461 minutes of audio from three habitats across nine locations, yielding 3,749 annotations for 34 classes.<n>Applying the Bird Song Detector before classification improved species identification, as all classification models performed better when analyzing only the segments where birds were detected.
  arXiv  Detail & Related papers  (2025-03-19T13:19:06Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts.<n>Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• NBM: an Open Dataset for the Acoustic Monitoring of Nocturnal Migratory Birds in Europe [0.0]
  This work presents the Nocturnal Bird Migration dataset, a collection of 13,359 annotated vocalizations from 117 species of the Western Palearctic.<n>The dataset includes precise time and frequency annotations, gathered by dozens of bird enthusiasts across France.<n>In particular, we prove the utility of this database by training an original two-stage deep object detection model tailored for the processing of audio data.
  arXiv  Detail & Related papers  (2024-12-04T18:55:45Z)
• Advanced Framework for Animal Sound Classification With Features Optimization [35.2832738406242]
  We propose an automated classification framework applicable to general animal sound classification. Our approach consistently outperforms baseline methods by over 25% in precision, recall, and accuracy.
  arXiv  Detail & Related papers  (2024-07-03T18:33:47Z)
• AudioProtoPNet: An interpretable deep learning model for bird sound classification [1.49199020343864]
  This study introduces AudioProtoPNet, an adaptation of the Prototypical Part Network (ProtoPNet) for multi-label bird sound classification. It is an inherently interpretable model that uses a ConvNeXt backbone to extract embeddings. The model was trained on the BirdSet training dataset, which consists of 9,734 bird species and over 6,800 hours of recordings.
  arXiv  Detail & Related papers  (2024-04-16T09:37:41Z)
• BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics [2.2399415927517414]
  textttBirdSet is a large-scale benchmark dataset for audio classification focusing on avian bioacoustics.<n>textttBirdSet surpasses AudioSet with over 6,800 recording hours($uparrow!17%$) from nearly 10,000 classes($uparrow!18times$) for training and more than 400 hours($uparrow!7times$) across eight strongly labeled evaluation datasets.
  arXiv  Detail & Related papers  (2024-03-15T15:10:40Z)
• Impact of Noisy Supervision in Foundation Model Learning [91.56591923244943]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets.<n>We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• WhaleNet: a Novel Deep Learning Architecture for Marine Mammals Vocalizations on Watkins Marine Mammal Sound Database [49.1574468325115]
  We introduce textbfWhaleNet (Wavelet Highly Adaptive Learning Ensemble Network), a sophisticated deep ensemble architecture for the classification of marine mammal vocalizations. We achieve an improvement in classification accuracy by $8-10%$ over existing architectures, corresponding to a classification accuracy of $97.61%$.
  arXiv  Detail & Related papers  (2024-02-20T11:36:23Z)
• Exploring Meta Information for Audio-based Zero-shot Bird Classification [113.17261694996051]
  This study investigates how meta-information can improve zero-shot audio classification. We use bird species as an example case study due to the availability of rich and diverse meta-data.
  arXiv  Detail & Related papers  (2023-09-15T13:50:16Z)
• Active Bird2Vec: Towards End-to-End Bird Sound Monitoring with Transformers [2.404305970432934]
  We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL) We aim to bypass traditional spectrogram conversions, enabling direct raw audio processing.
  arXiv  Detail & Related papers  (2023-08-14T13:06:10Z)
• ZooD: Exploiting Model Zoo for Out-of-Distribution Generalization [65.58562481279023]
  We propose ZooD, a paradigm for PTMs ranking and ensemble with feature selection. We evaluate our paradigm on a diverse model zoo consisting of 35 models for various Out-of-Distribution (OoD) tasks.
  arXiv  Detail & Related papers  (2022-10-17T16:31:57Z)
• Low-complexity deep learning frameworks for acoustic scene classification [64.22762153453175]
  We present low-complexity deep learning frameworks for acoustic scene classification (ASC) The proposed frameworks can be separated into four main steps: Front-end spectrogram extraction, online data augmentation, back-end classification, and late fusion of predicted probabilities. Our experiments conducted on DCASE 2022 Task 1 Development dataset have fullfiled the requirement of low-complexity and achieved the best classification accuracy of 60.1%.
  arXiv  Detail & Related papers  (2022-06-13T11:41:39Z)
• AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach [50.855679274530615]
  We present a novel domain-adaptive approach called AdaStereo to align multi-level representations for deep stereo matching networks. Our models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo. Our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.
  arXiv  Detail & Related papers  (2021-12-09T15:10:47Z)
• Training Classifiers that are Universally Robust to All Label Noise Levels [91.13870793906968]
  Deep neural networks are prone to overfitting in the presence of label noise. We propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. Our framework generally outperforms at medium to high noise levels.
  arXiv  Detail & Related papers  (2021-05-27T13:49:31Z)
• Discriminative Singular Spectrum Classifier with Applications on Bioacoustic Signal Recognition [67.4171845020675]
  We present a bioacoustic signal classifier equipped with a discriminative mechanism to extract useful features for analysis and classification efficiently. Unlike current bioacoustic recognition methods, which are task-oriented, the proposed model relies on transforming the input signals into vector subspaces. The validity of the proposed method is verified using three challenging bioacoustic datasets containing anuran, bee, and mosquito species.
  arXiv  Detail & Related papers  (2021-03-18T11:01:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.