What Matters for Bioacoustic Encoding

• URL: http://arxiv.org/abs/2508.11845v2
• Date: Tue, 19 Aug 2025 12:07:19 GMT
• Title: What Matters for Bioacoustic Encoding
• Authors: Marius Miron, David Robinson, Milad Alizadeh, Ellen Gilsenan-McMahon, Gagan Narula, Emmanuel Chemla, Maddie Cusimano, Felix Effenberger, Masato Hagiwara, Benjamin Hoffman, Sara Keen, Diane Kim, Jane Lawton, Jen-Yu Liu, Aza Raskin, Olivier Pietquin, Matthieu Geist,
• Abstract summary: We present a large-scale empirical study that covers aspects of bioacoustics that are relevant to research.<n>We obtain encoders that are state-of-the-art on the existing and proposed benchmarks.<n>Specifically, across 26 datasets with tasks including species classification, detection, individual ID, and vocal repertoire discovery, we find self-supervised pre-training followed by supervised post-training.
• Score: 34.118070876417065
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Bioacoustics, the study of sounds produced by living organisms, plays a vital role in conservation, biodiversity monitoring, and behavioral studies. Many tasks in this field, such as species, individual, and behavior classification and detection, are well-suited to machine learning. However, they often suffer from limited annotated data, highlighting the need for a general-purpose bioacoustic encoder capable of extracting useful representations for diverse downstream tasks. Such encoders have been proposed before, but are often limited in scope due to a focus on a narrow range of species (typically birds), and a reliance on a single model architecture or training paradigm. Moreover, they are usually evaluated on a small set of tasks and datasets. In this work, we present a large-scale empirical study that covers aspects of bioacoustics that are relevant to research but have previously been scarcely considered: training data diversity and scale, model architectures and training recipes, and the breadth of evaluation tasks and datasets. We obtain encoders that are state-of-the-art on the existing and proposed benchmarks. We also identify what matters for training these encoders, such that this work can be extended when more data are available or better architectures are proposed. Specifically, across 26 datasets with tasks including species classification, detection, individual ID, and vocal repertoire discovery, we find self-supervised pre-training followed by supervised post-training on a mixed bioacoustics + general-audio corpus yields the strongest in- and out-of-distribution performance. We show the importance of data diversity in both stages. To support ongoing research and application, we will release the model checkpoints.

Related papers

• The iNaturalist Sounds Dataset [60.157076990024606]
  iNatSounds is a collection of 230,000 audio files capturing sounds from over 5,500 species, contributed by more than 27,000 recordists worldwide.<n>The dataset encompasses sounds from birds, mammals, insects, reptiles, and amphibians, with audio and species labels derived from observations submitted to iNaturalist.<n>We envision models trained on this data powering next-generation public engagement applications, and assisting biologists, ecologists, and land use managers in processing large audio collections.
  arXiv  Detail & Related papers  (2025-05-31T02:07:37Z)
• NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics [35.72581102737726]
  We present NatureLM-audio, the first audio-language foundation model specifically designed for bioacoustics.<n>Our training dataset consists of carefully curated text-audio pairs spanning bioacoustics, speech, and music.<n>We demonstrate successful transfer of learned representations from music and speech to bioacoustics, and our model shows promising generalization to unseen taxa and tasks.
  arXiv  Detail & Related papers  (2024-11-11T18:01:45Z)
• Learning from Neighbors: Category Extrapolation for Long-Tail Learning [62.30734737735273]
  We offer a novel perspective on long-tail learning, inspired by an observation: datasets with finer granularity tend to be less affected by data imbalance.<n>We introduce open-set auxiliary classes that are visually similar to existing ones, aiming to enhance representation learning for both head and tail classes.<n>To prevent the overwhelming presence of auxiliary classes from disrupting training, we introduce a neighbor-silencing loss.
  arXiv  Detail & Related papers  (2024-10-21T13:06:21Z)
• Self-supervised Learning for Acoustic Few-Shot Classification [10.180992026994739]
  We introduce and evaluate a new architecture that combines CNN-based preprocessing with feature extraction based on state space models (SSMs)<n>We pre-train this architecture using contrastive learning on the actual task data and subsequent fine-tuning with an extremely small amount of labelled data.<n>Our evaluation shows that it outperforms state-of-the-art architectures on the few-shot classification problem.
  arXiv  Detail & Related papers  (2024-09-15T07:45:11Z)
• A Closer Look at Deep Learning Methods on Tabular Datasets [78.61845513154502]
  We present an extensive study on TALENT, a collection of 300+ datasets spanning broad ranges of size.<n>Our evaluation shows that ensembling benefits both tree-based and neural approaches.
  arXiv  Detail & Related papers  (2024-07-01T04:24:07Z)
• A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset [18.211840156134784]
  This paper presents a curated million-image dataset, primarily to train computer-vision models capable of providing image-based taxonomic assessment. The dataset also presents compelling characteristics, the study of which would be of interest to the broader machine learning community.
  arXiv  Detail & Related papers  (2023-07-19T20:54:08Z)
• USB: A Unified Summarization Benchmark Across Tasks and Domains [68.82726887802856]
  We introduce a Wikipedia-derived benchmark, complemented by a rich set of crowd-sourced annotations, that supports $8$ interrelated tasks. We compare various methods on this benchmark and discover that on multiple tasks, moderately-sized fine-tuned models consistently outperform much larger few-shot prompted language models.
  arXiv  Detail & Related papers  (2023-05-23T17:39:54Z)
• Representation Learning for the Automatic Indexing of Sound Effects Libraries [79.68916470119743]
  We show that a task-specific but dataset-independent representation can successfully address data issues such as class imbalance, inconsistent class labels, and insufficient dataset size. Detailed experimental results show the impact of metric learning approaches and different cross-dataset training methods on representational effectiveness.
  arXiv  Detail & Related papers  (2022-08-18T23:46:13Z)
• Learning Generic Lung Ultrasound Biomarkers for Decoupling Feature Extraction from Downstream Tasks [0.032270246323516584]
  We propose to decouple feature learning from downstream lung ultrasound tasks by introducing an auxiliary pre-task of visual biomarker classification. We demonstrate that one can learn an informative, concise, and interpretable feature space from ultrasound videos by training models for predicting biomarker labels.
  arXiv  Detail & Related papers  (2022-06-16T18:15:14Z)
• Parsing Birdsong with Deep Audio Embeddings [0.5599792629509227]
  We present a semi-supervised approach to identify characteristic calls and environmental noise. We utilize several methods to learn a latent representation of audio samples, including a convolutional autoencoder and two pre-trained networks.
  arXiv  Detail & Related papers  (2021-08-20T14:45:44Z)
• Diverse Complexity Measures for Dataset Curation in Self-driving [80.55417232642124]
  We propose a new data selection method that exploits a diverse set of criteria that quantize interestingness of traffic scenes. Our experiments show that the proposed curation pipeline is able to select datasets that lead to better generalization and higher performance.
  arXiv  Detail & Related papers  (2021-01-16T23:45:02Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.