Automated Behavioral Analysis Using Instance Segmentation
- URL: http://arxiv.org/abs/2312.07723v1
- Date: Tue, 12 Dec 2023 20:36:36 GMT
- Title: Automated Behavioral Analysis Using Instance Segmentation
- Authors: Chen Yang, Jeremy Forest, Matthew Einhorn, Thomas A. Cleland
- Abstract summary: Animal behavior analysis plays a crucial role in various fields, such as life science and biomedical research.
The scarcity of available data and the high cost associated with obtaining a large number of labeled datasets pose significant challenges.
We propose a novel approach that leverages instance segmentation-based transfer learning to address these issues.
- Score: 2.043437148047176
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Animal behavior analysis plays a crucial role in various fields, such as life
science and biomedical research. However, the scarcity of available data and
the high cost associated with obtaining a large number of labeled datasets pose
significant challenges. In this research, we propose a novel approach that
leverages instance segmentation-based transfer learning to address these
issues. By capitalizing on fine-tuning the classification head of the instance
segmentation network, we enable the tracking of multiple animals and facilitate
behavior analysis in laboratory-recorded videos. To demonstrate the
effectiveness of our method, we conducted a series of experiments, revealing
that our approach achieves exceptional performance levels, comparable to human
capabilities, across a diverse range of animal behavior analysis tasks.
Moreover, we emphasize the practicality of our solution, as it requires only a
small number of labeled images for training. To facilitate the adoption and
further development of our method, we have developed an open-source
implementation named Annolid (An annotation and instance segmentation-based
multiple animal tracking and behavior analysis package). The codebase is
publicly available on GitHub at https://github.com/cplab/annolid. This resource
serves as a valuable asset for researchers and practitioners interested in
advancing animal behavior analysis through state-of-the-art techniques.
Related papers
- Annolid: Annotate, Segment, and Track Anything You Need [2.0517786943861376]
Annolid is a software package designed for the segmentation, labeling, and tracking of research targets within video files.
Based on state-of-the-art instance segmentation methods, Annolid now harnesses the Cutie video object segmentation model.
Our integration of Segment Anything and Grounding-DINO strategies additionally enables the automatic masking and segmentation of recognizable animals and objects by text command.
arXiv Detail & Related papers (2024-03-27T15:41:23Z) - Computer Vision for Primate Behavior Analysis in the Wild [61.08941894580172]
Video-based behavioral monitoring has great potential for transforming how we study animal cognition and behavior.
There is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today.
arXiv Detail & Related papers (2024-01-29T18:59:56Z) - A benchmark for computational analysis of animal behavior, using animal-borne tags [0.6055654308018396]
We present the Bio-logger Ethogram Benchmark (BEBE), a collection of datasets with behavioral annotations.
We compare the performance of deep and classical machine learning methods for identifying animal behaviors based on bio-logger data.
arXiv Detail & Related papers (2023-05-18T06:20:45Z) - CNN-Based Action Recognition and Pose Estimation for Classifying Animal
Behavior from Videos: A Survey [0.0]
Action recognition, classifying activities performed by one or more subjects in a trimmed video, forms the basis of many techniques.
Deep learning models for human action recognition have progressed over the last decade.
Recent interest in research that incorporates deep learning-based action recognition for classification has increased.
arXiv Detail & Related papers (2023-01-15T20:54:44Z) - Review on Social Behavior Analysis of Laboratory Animals: From
Methodologies to Applications [0.0]
We explore a variety of behaviour detection algorithms, covering traditional vision methods, statistical methods and deep learning methods.
The objective of this work is to provide a thorough investigation of related work, furnishing biologists with a scratch of efficient animal behaviour detection methods.
arXiv Detail & Related papers (2022-06-25T13:40:35Z) - Animal Kingdom: A Large and Diverse Dataset for Animal Behavior
Understanding [4.606145900630665]
We create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks.
Our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments.
We propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals.
arXiv Detail & Related papers (2022-04-18T02:05:15Z) - Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
3D pose data can now be reliably extracted from multi-view video sequences without manual intervention.
We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations.
To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
arXiv Detail & Related papers (2021-12-02T12:45:46Z) - Cetacean Translation Initiative: a roadmap to deciphering the
communication of sperm whales [97.41394631426678]
Recent research showed the promise of machine learning tools for analyzing acoustic communication in nonhuman species.
We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales.
The technological capabilities developed are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
arXiv Detail & Related papers (2021-04-17T18:39:22Z) - A Trainable Optimal Transport Embedding for Feature Aggregation and its
Relationship to Attention [96.77554122595578]
We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference.
Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
arXiv Detail & Related papers (2020-06-22T08:35:58Z) - Extending and Analyzing Self-Supervised Learning Across Domains [50.13326427158233]
Self-supervised representation learning has achieved impressive results in recent years.
Experiments primarily come on ImageNet or other similarly large internet imagery datasets.
We experiment with several popular methods on an unprecedented variety of domains.
arXiv Detail & Related papers (2020-04-24T21:18:02Z) - Transferring Dense Pose to Proximal Animal Classes [83.84439508978126]
We show that it is possible to transfer the knowledge existing in dense pose recognition for humans, as well as in more general object detectors and segmenters, to the problem of dense pose recognition in other classes.
We do this by establishing a DensePose model for the new animal which is also geometrically aligned to humans.
We also introduce two benchmark datasets labelled in the manner of DensePose for the class chimpanzee and use them to evaluate our approach.
arXiv Detail & Related papers (2020-02-28T21:43:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.