AudioProtoPNet: An interpretable deep learning model for bird sound classification

• URL: http://arxiv.org/abs/2404.10420v2
• Date: Wed, 29 May 2024 14:09:17 GMT
• Title: AudioProtoPNet: An interpretable deep learning model for bird sound classification
• Authors: René Heinrich, Bernhard Sick, Christoph Scholz,
• Abstract summary: We present an adaption of the Prototypical Part Network (ProtoPNet) for audio classification that provides inherent interpretability through its model architecture. Our approach is based on a ConvNeXt backbone architecture for feature extraction and learns patterns for each bird species using spectrograms of the training data.
• Score: 1.6298921134113031
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, scientists have proposed several deep learning models to monitor the diversity of bird species. These models can detect bird species with high accuracy by analyzing acoustic signals. However, traditional deep learning algorithms are black-box models that provide no insight into their decision-making process. For domain experts, such as ornithologists, it is crucial that these models are not only efficient, but also interpretable in order to be used as assistive tools. In this study, we present an adaption of the Prototypical Part Network (ProtoPNet) for audio classification that provides inherent interpretability through its model architecture. Our approach is based on a ConvNeXt backbone architecture for feature extraction and learns prototypical patterns for each bird species using spectrograms of the training data. Classification of new data is done by comparison with these prototypes in latent space, which simultaneously serve as easily understandable explanations for the model's decisions. We evaluated the performance of our model on seven different datasets representing bird species from different geographical regions. In our experiments, the model showed excellent results, achieving an average AUROC of 0.82 and an average cmAP of 0.37 across the seven datasets, making it comparable to state-of-the-art black-box models for bird sound classification. Thus, this work demonstrates that even for the challenging task of bioacoustic bird classification, powerful yet interpretable deep learning models can be developed to provide valuable insights to domain experts.

Related papers

• TinyChirp: Bird Song Recognition Using TinyML Models on Low-power Wireless Acoustic Sensors [1.0790796076947324]
  Monitoring biodiversity at scale is challenging. detecting and identifying species in fine grained requires highly accurate machine learning (ML) methods. deploying these models to low power devices requires novel compression techniques and model architectures.
  arXiv  Detail & Related papers  (2024-07-31T08:57:42Z)
• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Comparison of self-supervised in-domain and supervised out-domain transfer learning for bird species recognition [0.19183348587701113]
  Transferring the weights of a pre-trained model to assist another task has become a crucial part of modern deep learning. Our experiments will demonstrate the usefulness of in-domain models and datasets for bird species recognition.
  arXiv  Detail & Related papers  (2024-04-26T08:47:28Z)
• Auto deep learning for bioacoustic signals [2.833479881983341]
  This study investigates the potential of automated deep learning to enhance the accuracy and efficiency of multi-class classification of bird vocalizations. Using the Western Mediterranean Wetland Birds dataset, we investigated the use of AutoKeras, an automated machine learning framework.
  arXiv  Detail & Related papers  (2023-11-08T07:22:39Z)
• Diffusion Models Beat GANs on Image Classification [37.70821298392606]
  Diffusion models have risen to prominence as a state-of-the-art method for image generation, denoising, inpainting, super-resolution, manipulation, etc. We present our findings that these embeddings are useful beyond the noise prediction task, as they contain discriminative information and can also be leveraged for classification. We find that with careful feature selection and pooling, diffusion models outperform comparable generative-discriminative methods for classification tasks.
  arXiv  Detail & Related papers  (2023-07-17T17:59:40Z)
• Knowledge is a Region in Weight Space for Fine-tuned Language Models [48.589822853418404]
  We study how the weight space and the underlying loss landscape of different models are interconnected. We show that language models that have been finetuned on the same dataset form a tight cluster in the weight space, while models finetuned on different datasets from the same underlying task form a looser cluster.
  arXiv  Detail & Related papers  (2023-02-09T18:59:18Z)
• An empirical investigation into audio pipeline approaches for classifying bird species [0.9158130615768508]
  This paper is an investigation into aspects of an audio classification pipeline that will be appropriate for the monitoring of bird species on edges devices. Two classification approaches will be taken into consideration, one which explores the effectiveness of a traditional Deep Neural Network(DNN) and another that makes use of Convolutional layers.
  arXiv  Detail & Related papers  (2021-08-10T05:02:38Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Improving Label Quality by Jointly Modeling Items and Annotators [68.8204255655161]
  We propose a fully Bayesian framework for learning ground truth labels from noisy annotators. Our framework ensures scalability by factoring a generative, Bayesian soft clustering model over label distributions into the classic David and Skene joint annotator-data model.
  arXiv  Detail & Related papers  (2021-06-20T02:15:20Z)
• Visualising Deep Network's Time-Series Representations [93.73198973454944]
  Despite the popularisation of machine learning models, more often than not they still operate as black boxes with no insight into what is happening inside the model. In this paper, a method that addresses that issue is proposed, with a focus on visualising multi-dimensional time-series data. Experiments on a high-frequency stock market dataset show that the method provides fast and discernible visualisations.
  arXiv  Detail & Related papers  (2021-03-12T09:53:34Z)
• Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning [109.74041512359476]
  We study a number of design decisions for the predictive model in visual MBRL algorithms. We find that a range of design decisions that are often considered crucial, such as the use of latent spaces, have little effect on task performance. We show how this phenomenon is related to exploration and how some of the lower-scoring models on standard benchmarks will perform the same as the best-performing models when trained on the same training data.
  arXiv  Detail & Related papers  (2020-12-08T18:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.