Recognition of Unseen Bird Species by Learning from Field Guides

• URL: http://arxiv.org/abs/2206.01466v2
• Date: Thu, 2 Nov 2023 18:03:46 GMT
• Title: Recognition of Unseen Bird Species by Learning from Field Guides
• Authors: Andr\'es C. Rodr\'iguez, Stefano D'Aronco, Rodrigo Caye Daudt, Jan D. Wegner, Konrad Schindler
• Abstract summary: We exploit field guides to learn bird species recognition, in particular zero-shot recognition of unseen species. We study two approaches: (1) a contrastive encoding of illustrations, which can be fed into standard zero-shot learning schemes; and (2) a novel method that leverages the fact that illustrations are also images. Our results show that illustrations from field guides, which are readily available for a wide range of species, are indeed a competitive source of side information for zero-shot learning.
• Score: 23.137536032163855
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We exploit field guides to learn bird species recognition, in particular zero-shot recognition of unseen species. Illustrations contained in field guides deliberately focus on discriminative properties of each species, and can serve as side information to transfer knowledge from seen to unseen bird species. We study two approaches: (1) a contrastive encoding of illustrations, which can be fed into standard zero-shot learning schemes; and (2) a novel method that leverages the fact that illustrations are also images and as such structurally more similar to photographs than other kinds of side information. Our results show that illustrations from field guides, which are readily available for a wide range of species, are indeed a competitive source of side information for zero-shot learning. On a subset of the iNaturalist2021 dataset with 749 seen and 739 unseen species, we obtain a classification accuracy of unseen bird species of $12\%$ @top-1 and $38\%$ @top-10, which shows the potential of field guides for challenging real-world scenarios with many species. Our code is available at https://github.com/ac-rodriguez/zsl_billow

Related papers

• Leveraging Habitat Information for Fine-grained Bird Identification [4.392299539811761]
  We are the first to explore integrating habitat information, one of the four major cues for identifying birds by ornithologists, into modern bird classifiers. We focus on two leading model types: CNNs and ViTs trained on the downstream bird datasets; and original, multi-modal CLIP. Training CNNs and ViTs with habitat-augmented data results in an improvement of up to +0.83 and +0.23 points on NABirds and CUB-200, respectively.
  arXiv  Detail & Related papers  (2023-12-22T16:23:22Z)
• Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images [57.96659470133514]
  Motion-activated camera traps constitute an efficient tool for tracking and monitoring wildlife populations across the globe. Supervised learning techniques have been successfully deployed to analyze such imagery, however training such techniques requires annotations from experts. Reducing the reliance on costly labelled data has immense potential in developing large-scale wildlife tracking solutions with markedly less human labor.
  arXiv  Detail & Related papers  (2023-11-02T08:32:00Z)
• Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognition [30.327642724046903]
  Species196 is a large-scale semi-supervised dataset of 196-category invasive species. It collects over 19K images with expert-level accurate annotations Species196-L, and 1.2M unlabeled images of invasive species Species196-U.
  arXiv  Detail & Related papers  (2023-09-25T14:46:01Z)
• Spatial Implicit Neural Representations for Global-Scale Species Mapping [72.92028508757281]
  Given a set of locations where a species has been observed, the goal is to build a model to predict whether the species is present or absent at any location. Traditional methods struggle to take advantage of emerging large-scale crowdsourced datasets. We use Spatial Implicit Neural Representations (SINRs) to jointly estimate the geographical range of 47k species simultaneously.
  arXiv  Detail & Related papers  (2023-06-05T03:36:01Z)
• Florida Wildlife Camera Trap Dataset [48.99466876948454]
  We introduce a challenging wildlife camera trap classification dataset collected from two different locations in Southwestern Florida. The dataset consists of 104,495 images featuring visually similar species, varying illumination conditions, skewed class distribution, and including samples of endangered species.
  arXiv  Detail & Related papers  (2021-06-23T18:53:15Z)
• Birds of a Feather: Capturing Avian Shape Models from Images [46.84613650829397]
  We present a method to capture new species using an articulated template and images of that species. We learn a shape space that captures variation both among species and within each species from image evidence. Using a low-dimensional embedding, we show that our learned 3D shape space better reflects the phylogenetic relationships among birds than learned perceptual features.
  arXiv  Detail & Related papers  (2021-05-19T20:53:48Z)
• One-shot learning for acoustic identification of bird species in non-stationary environments [5.177947445379688]
  We propose a framework able to detect changes in the class dictionary and incorporate new classes on the fly. We design an one-shot learning architecture composed of a Siamese Neural Network operating in the logMel spectrogram space.
  arXiv  Detail & Related papers  (2021-05-01T09:43:20Z)
• Two-View Fine-grained Classification of Plant Species [66.75915278733197]
  We propose a novel method based on a two-view leaf image representation and a hierarchical classification strategy for fine-grained recognition of plant species. A deep metric based on Siamese convolutional neural networks is used to reduce the dependence on a large number of training samples and make the method scalable to new plant species.
  arXiv  Detail & Related papers  (2020-05-18T21:57:47Z)
• Transferring Dense Pose to Proximal Animal Classes [83.84439508978126]
  We show that it is possible to transfer the knowledge existing in dense pose recognition for humans, as well as in more general object detectors and segmenters, to the problem of dense pose recognition in other classes. We do this by establishing a DensePose model for the new animal which is also geometrically aligned to humans. We also introduce two benchmark datasets labelled in the manner of DensePose for the class chimpanzee and use them to evaluate our approach.
  arXiv  Detail & Related papers  (2020-02-28T21:43:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.