Detecting Endangered Marine Species in Autonomous Underwater Vehicle Imagery Using Point Annotations and Few-Shot Learning

• URL: http://arxiv.org/abs/2406.01932v1
• Date: Tue, 4 Jun 2024 03:31:42 GMT
• Title: Detecting Endangered Marine Species in Autonomous Underwater Vehicle Imagery Using Point Annotations and Few-Shot Learning
• Authors: Heather Doig, Oscar Pizarro, Jacquomo Monk, Stefan Williams,
• Abstract summary: Seafloor imagery collected by Autonomous Underwater Vehicles (AUVs) can be used to identify individuals within their broader habitat context. Machine learning models can be used to identify the presence of a particular species in images using a trained object detector. In this paper, inspired by recent work in few-shot learning, images and annotations of common marine species are exploited to enhance the ability of the detector to identify rare and cryptic species.
• Score: 5.439798554380394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One use of Autonomous Underwater Vehicles (AUVs) is the monitoring of habitats associated with threatened, endangered and protected marine species, such as the handfish of Tasmania, Australia. Seafloor imagery collected by AUVs can be used to identify individuals within their broader habitat context, but the sheer volume of imagery collected can overwhelm efforts to locate rare or cryptic individuals. Machine learning models can be used to identify the presence of a particular species in images using a trained object detector, but the lack of training examples reduces detection performance, particularly for rare species that may only have a small number of examples in the wild. In this paper, inspired by recent work in few-shot learning, images and annotations of common marine species are exploited to enhance the ability of the detector to identify rare and cryptic species. Annotated images of six common marine species are used in two ways. Firstly, the common species are used in a pre-training step to allow the backbone to create rich features for marine species. Secondly, a copy-paste operation is used with the common species images to augment the training data. While annotations for more common marine species are available in public datasets, they are often in point format, which is unsuitable for training an object detector. A popular semantic segmentation model efficiently generates bounding box annotations for training from the available point annotations. Our proposed framework is applied to AUV images of handfish, increasing average precision by up to 48\% compared to baseline object detection training. This approach can be applied to other objects with low numbers of annotations and promises to increase the ability to actively monitor threatened, endangered and protected species.

Related papers

• Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs [31.22129440376567]
  We exploit the structured context linked to camera trap images to boost out-of-distribution generalization for species classification tasks in camera traps. A picture of a wild animal could be linked to details about the time and place it was captured, as well as structured biological knowledge about the animal species. We propose a novel framework that transforms species classification as link prediction in a multimodal knowledge graph.
  arXiv  Detail & Related papers  (2023-12-31T23:32:03Z)
• Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images [57.96659470133514]
  Motion-activated camera traps constitute an efficient tool for tracking and monitoring wildlife populations across the globe. Supervised learning techniques have been successfully deployed to analyze such imagery, however training such techniques requires annotations from experts. Reducing the reliance on costly labelled data has immense potential in developing large-scale wildlife tracking solutions with markedly less human labor.
  arXiv  Detail & Related papers  (2023-11-02T08:32:00Z)
• Generalized Few-Shot 3D Object Detection of LiDAR Point Cloud for Autonomous Driving [91.39625612027386]
  We propose a novel task, called generalized few-shot 3D object detection, where we have a large amount of training data for common (base) objects, but only a few data for rare (novel) classes. Specifically, we analyze in-depth differences between images and point clouds, and then present a practical principle for the few-shot setting in the 3D LiDAR dataset. To solve this task, we propose an incremental fine-tuning method to extend existing 3D detection models to recognize both common and rare objects.
  arXiv  Detail & Related papers  (2023-02-08T07:11:36Z)
• Deep Model Intellectual Property Protection via Deep Watermarking [122.87871873450014]
  Deep neural networks are exposed to serious IP infringement risks. Given a target deep model, if the attacker knows its full information, it can be easily stolen by fine-tuning. We propose a new model watermarking framework for protecting deep networks trained for low-level computer vision or image processing tasks.
  arXiv  Detail & Related papers  (2021-03-08T18:58:21Z)
• Movement Tracks for the Automatic Detection of Fish Behavior in Videos [63.85815474157357]
  We offer a dataset of sablefish (Anoplopoma fimbria) startle behaviors in underwater videos, and investigate the use of deep learning (DL) methods for behavior detection on it. Our proposed detection system identifies fish instances using DL-based frameworks, determines trajectory tracks, derives novel behavior-specific features, and employs Long Short-Term Memory (LSTM) networks to identify startle behavior in sablefish.
  arXiv  Detail & Related papers  (2020-11-28T05:51:19Z)
• Unifying data for fine-grained visual species classification [15.14767769034929]
  We present an initial deep convolutional neural network model, trained on 2.9M images across 465 fine-grained species. The long-term goal is to enable scientists to make conservation recommendations from near real-time analysis of species abundance and population health.
  arXiv  Detail & Related papers  (2020-09-24T01:04:18Z)
• Temperate Fish Detection and Classification: a Deep Learning based Approach [6.282069822653608]
  We propose a two-step deep learning approach for the detection and classification of temperate fishes without pre-filtering. The first step is to detect each single fish in an image, independent of species and sex. In the second step, we adopt a Convolutional Neural Network (CNN) with the Squeeze-and-Excitation (SE) architecture for classifying each fish in the image without pre-filtering.
  arXiv  Detail & Related papers  (2020-05-14T12:40:57Z)
• Automatic Detection and Recognition of Individuals in Patterned Species [4.163860911052052]
  We develop a framework for automatic detection and recognition of individuals in different patterned species. We use the recently proposed Faster-RCNN object detection framework to efficiently detect animals in images. We evaluate our recognition system on zebra and jaguar images to show generalization to other patterned species.
  arXiv  Detail & Related papers  (2020-05-06T15:29:21Z)
• Fine-grained Species Recognition with Privileged Pooling: Better Sample Efficiency Through Supervised Attention [26.136331738529243]
  We propose a scheme for supervised image classification that uses privileged information in the form of keypoint annotations for the training data. Our main motivation is the recognition of animal species for ecological applications such as biodiversity modelling. In experiments with three different animal species datasets, we show that deep networks with privileged pooling can use small training sets more efficiently and generalize better.
  arXiv  Detail & Related papers  (2020-03-20T10:03:01Z)
• Transferring Dense Pose to Proximal Animal Classes [83.84439508978126]
  We show that it is possible to transfer the knowledge existing in dense pose recognition for humans, as well as in more general object detectors and segmenters, to the problem of dense pose recognition in other classes. We do this by establishing a DensePose model for the new animal which is also geometrically aligned to humans. We also introduce two benchmark datasets labelled in the manner of DensePose for the class chimpanzee and use them to evaluate our approach.
  arXiv  Detail & Related papers  (2020-02-28T21:43:53Z)
• Model Watermarking for Image Processing Networks [120.918532981871]
  How to protect the intellectual property of deep models is a very important but seriously under-researched problem. We propose the first model watermarking framework for protecting image processing models.
  arXiv  Detail & Related papers  (2020-02-25T18:36:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.