Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics

• URL: http://arxiv.org/abs/2404.16436v2
• Date: Tue, 7 May 2024 12:42:32 GMT
• Title: Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics
• Abstract summary: Machine learning has the potential to revolutionize passive acoustic monitoring (PAM) for ecological assessments. Generalizable pretrained networks can overcome these costs, but high-quality pretraining requires vast annotated libraries. Here, we identify the optimum pretraining strategy for a data-deficient domain using coral reef bioacoustics.
• Score: 5.799126142155481
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning has the potential to revolutionize passive acoustic monitoring (PAM) for ecological assessments. However, high annotation and compute costs limit the field's efficacy. Generalizable pretrained networks can overcome these costs, but high-quality pretraining requires vast annotated libraries, limiting its current applicability primarily to bird taxa. Here, we identify the optimum pretraining strategy for a data-deficient domain using coral reef bioacoustics. We assemble ReefSet, a large annotated library of reef sounds, though modest compared to bird libraries at 2% of the sample count. Through testing few-shot transfer learning performance, we observe that pretraining on bird audio provides notably superior generalizability compared to pretraining on ReefSet or unrelated audio alone. However, our key findings show that cross-domain mixing which leverages bird, reef and unrelated audio during pretraining maximizes reef generalizability. SurfPerch, our pretrained network, provides a strong foundation for automated analysis of marine PAM data with minimal annotation and compute costs.

Related papers

• MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training [7.983925207958734]
  Time-frequency annotation of animal vocalizations is expensive, site-specific, and difficult to sustain at scale.<n>We present a label-efficient sound detection framework that combines masked audio pretraining with a lightweight detector on mel spectrograms.<n>We evaluate the performance on two ecologically distinct domains: tropical rainforest soundscapes (Indonesia) and bird vocalizations in Mediterranean habitats (Spain)
  arXiv  Detail & Related papers  (2026-09-14T08:41:23Z)
• Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study [20.469464200788583]
  Masked autoencoders (MAE) have shown strong transferability on massive audio corpora.<n>We conduct a systematic study of MAE pretraining for species classification on iNatSounds.<n>Our results indicate that, in moderate-sized fine-grained bioacoustic settings, pretraining scale dominates objective design.
  arXiv  Detail & Related papers  (2026-05-13T18:45:08Z)
• NoiseRater: Meta-Learned Noise Valuation for Diffusion Model Training [81.58321000203712]
  NoiseRater is a meta-learning framework for instance-level noise valuation in diffusion model training.<n>NoiseRater assigns importance scores to individual noise realizations conditioned on data and timestep.<n>Experiments on FFHQ and ImageNet demonstrate that not all noise samples contribute equally.
  arXiv  Detail & Related papers  (2026-05-02T19:43:16Z)
• Decodable but not structured: linear probing enables Underwater Acoustic Target Recognition with pretrained audio embeddings [1.25052154879199]
  anthropogenic noise from ships contribute significantly to underwater sound pollution, posing risks to marine ecosystems.<n> Passive Acoustic Monitoring (PAM) systems are widely deployed for this purpose, generating years of underwater recordings across diverse soundscapes.<n>Recent advances in automatic Underwater Acoustic Target Recognition (UATR) have largely relied on supervised learning, which is constrained by the scarcity of labeled data.<n>In this work, we conduct the first empirical comparative study of transfer learning for UATR, evaluating multiple pretrained audio models originating from diverse audio domains.
  arXiv  Detail & Related papers  (2026-01-13T09:15:31Z)
• Learning Robust Spatial Representations from Binaural Audio through Feature Distillation [64.36563387033921]
  We investigate the use of a pretraining stage based on feature distillation to learn a robust spatial representation of speech without the need for data labels.<n>Our experiments demonstrate that the pretrained models show improved performance in noisy and reverberant environments.
  arXiv  Detail & Related papers  (2025-08-28T15:43:15Z)
• Perch 2.0: The Bittern Lesson for Bioacoustics [7.540404494602192]
  Perch is a performant pre-trained model for bioacoustics.<n>It was trained in supervised fashion, providing both off-the-shelf classification scores for thousands of vocalizing species as well as strong embeddings for transfer learning.<n>In this new release, Perch 2.0, we expand from training exclusively on avian species to a large multi-taxa dataset.
  arXiv  Detail & Related papers  (2025-08-06T17:34:43Z)
• Foundation Models for Bioacoustics -- a Comparative Review [0.9109149174920012]
  We review bioacoustic foundation models by thoroughly analysing design decisions such as model architecture, pretraining scheme, and training paradigm.<n>We evaluate selected foundation models on classification tasks from the BEANS and BirdSet benchmarks.<n>Our comprehensive experimental analysis reveals that BirdMAE, trained on large-scale bird song data with a self-supervised objective, achieves the best performance on the BirdSet benchmark.
  arXiv  Detail & Related papers  (2025-08-02T09:15:16Z)
• U-DREAM: Unsupervised Dereverberation guided by a Reverberation Model [12.192022160630165]
  This paper explores the outcome of training state-ofthe-art dereverberation models with supervision settings ranging from weakly-supervised to fully unsupervised.<n>Most of the existing deep learning approaches typically require paired dry and reverberant data, which are difficult to obtain in practice.<n>We develop instead a sequential learning strategy motivated by a bayesian formulation of the dereverberation problem, wherein acoustic parameters and dry signals are estimated from reverberant inputs using deep neural networks.
  arXiv  Detail & Related papers  (2025-07-17T12:26:18Z)
• Can Masked Autoencoders Also Listen to Birds? [2.430300340530418]
  Masked Autoencoders (MAEs) have shown competitive results in audio classification by learning rich semantic representations.<n>General-purpose models fail to generalize well when applied directly to fine-grained audio domains.<n>This work demonstrates that bridging this domain gap requires more than domain-specific pretraining data.
  arXiv  Detail & Related papers  (2025-04-17T12:13:25Z)
• Semi-supervised classification of bird vocalizations [0.0]
  Changes in bird populations can indicate broader changes in ecosystems. We propose a semi-supervised acoustic bird detector to allow the detection of time-overlapping calls. It achieves a mean F0.5 score of 0.701 across 315 classes from 110 bird species on a hold-out test set.
  arXiv  Detail & Related papers  (2025-02-19T05:31:13Z)
• Provably Efficient RLHF Pipeline: A Unified View from Contextual Bandits [59.30310692855397]
  We propose a unified framework for the RLHF pipeline from the view of contextual bandits. We decompose the RLHF process into two distinct stages: (post-)training and deployment. We then develop novel algorithms for each stage, demonstrating significant improvements in both statistical and computational efficiency.
  arXiv  Detail & Related papers  (2025-02-11T02:36:01Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Learning with Noisy Foundation Models [95.50968225050012]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets. We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• WhaleNet: a Novel Deep Learning Architecture for Marine Mammals Vocalizations on Watkins Marine Mammal Sound Database [49.1574468325115]
  We introduce textbfWhaleNet (Wavelet Highly Adaptive Learning Ensemble Network), a sophisticated deep ensemble architecture for the classification of marine mammal vocalizations. We achieve an improvement in classification accuracy by $8-10%$ over existing architectures, corresponding to a classification accuracy of $97.61%$.
  arXiv  Detail & Related papers  (2024-02-20T11:36:23Z)
• Self-Supervised Learning for Few-Shot Bird Sound Classification [10.395255631261458]
  Self-supervised learning (SSL) in audio holds significant potential across various domains. In this study, we demonstrate that SSL is capable of acquiring meaningful representations of bird sounds from audio recordings without the need for annotations.
  arXiv  Detail & Related papers  (2023-12-25T22:33:45Z)
• Noisy Self-Training with Synthetic Queries for Dense Retrieval [49.49928764695172]
  We introduce a novel noisy self-training framework combined with synthetic queries. Experimental results show that our method improves consistently over existing methods. Our method is data efficient and outperforms competitive baselines.
  arXiv  Detail & Related papers  (2023-11-27T06:19:50Z)
• Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks [91.15120211190519]
  This paper aims to understand the nature of noise in pre-training datasets and to mitigate its impact on downstream tasks. We propose a light-weight black-box tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise.
  arXiv  Detail & Related papers  (2023-09-29T06:18:15Z)
• Exploring Meta Information for Audio-based Zero-shot Bird Classification [113.17261694996051]
  This study investigates how meta-information can improve zero-shot audio classification. We use bird species as an example case study due to the availability of rich and diverse meta-data.
  arXiv  Detail & Related papers  (2023-09-15T13:50:16Z)
• Efficient acoustic feature transformation in mismatched environments using a Guided-GAN [1.495380389108477]
  We propose a new framework to improve automatic speech recognition systems in resource-scarce environments. We use a generative adversarial network (GAN) operating on acoustic input features to enhance the features of mismatched data. With less than one hour of data, an ASR system trained on good quality data, and evaluated on mismatched audio is improved by between 11.5% and 19.7% relative word error rate (WER)
  arXiv  Detail & Related papers  (2022-10-03T05:33:28Z)
• Classification of animal sounds in a hyperdiverse rainforest using Convolutional Neural Networks [0.0]
  Automated species detection from passively recorded soundscapes via machine-learning approaches is a promising technique. We use soundscapes from a tropical forest in Borneo and a Convolutional Neural Network model (CNN) created with transfer learning. Our results suggest that transfer learning and data augmentation can make the use of CNNs to classify species' vocalizations feasible even for small soundscape-based projects with many rare species.
  arXiv  Detail & Related papers  (2021-11-29T21:34:57Z)
• Recognizing bird species in diverse soundscapes under weak supervision [0.2148535041822524]
  We present a robust classification approach for avian vocalization in complex and diverse soundscapes, achieving second place in the BirdCLEF 2021 challenge. We illustrate how to make full use of pre-trained convolutional neural networks, by using an efficient modeling and training routine supplemented by novel augmentation methods.
  arXiv  Detail & Related papers  (2021-07-16T06:54:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.