Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding

• URL: http://arxiv.org/abs/2204.08129v1
• Date: Mon, 18 Apr 2022 02:05:15 GMT
• Title: Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding
• Authors: Xun Long Ng, Kian Eng Ong, Qichen Zheng, Yun Ni, Si Yong Yeo, Jun Liu
• Abstract summary: We create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks. Our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments. We propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals.
• Score: 4.606145900630665
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding animals' behaviors is significant for a wide range of applications. However, existing animal behavior datasets have limitations in multiple aspects, including limited numbers of animal classes, data samples and provided tasks, and also limited variations in environmental conditions and viewpoints. To address these limitations, we create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors. The wild animal footages used in our dataset record different times of the day in extensive range of environments containing variations in backgrounds, viewpoints, illumination and weather conditions. More specifically, our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments in long videos for the video grounding task, 30K video sequences for the fine-grained multi-label action recognition task, and 33K frames for the pose estimation task, which correspond to a diverse range of animals with 850 species across 6 major animal classes. Such a challenging and comprehensive dataset shall be able to facilitate the community to develop, adapt, and evaluate various types of advanced methods for animal behavior analysis. Moreover, we propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals. This method achieves promising performance in our experiments. Our dataset can be found at https://sutdcv.github.io/Animal-Kingdom.

Related papers

• From Forest to Zoo: Great Ape Behavior Recognition with ChimpBehave [0.0]
  We introduce ChimpBehave, a novel dataset featuring over 2 hours of video (approximately 193,000 video frames) of zoo-housed chimpanzees. ChimpBehave meticulously annotated with bounding boxes and behavior labels for action recognition. We benchmark our dataset using a state-of-the-art CNN-based action recognition model.
  arXiv  Detail & Related papers  (2024-05-30T13:11:08Z)
• AnimateZoo: Zero-shot Video Generation of Cross-Species Animation via Subject Alignment [64.02822911038848]
  We present AnimateZoo, a zero-shot diffusion-based video generator to produce animal animations. Key technique used in our AnimateZoo is subject alignment, which includes two steps. Our model is capable of generating videos characterized by accurate movements, consistent appearance, and high-fidelity frames.
  arXiv  Detail & Related papers  (2024-04-07T12:57:41Z)
• Automated Behavioral Analysis Using Instance Segmentation [2.043437148047176]
  Animal behavior analysis plays a crucial role in various fields, such as life science and biomedical research. The scarcity of available data and the high cost associated with obtaining a large number of labeled datasets pose significant challenges. We propose a novel approach that leverages instance segmentation-based transfer learning to address these issues.
  arXiv  Detail & Related papers  (2023-12-12T20:36:36Z)
• OmniMotionGPT: Animal Motion Generation with Limited Data [70.35662376853163]
  We introduce AnimalML3D, the first text-animal motion dataset with 1240 animation sequences spanning 36 different animal identities. We are able to generate animal motions with high diversity and fidelity, quantitatively and qualitatively outperforming the results of training human motion generation baselines on animal data.
  arXiv  Detail & Related papers  (2023-11-30T07:14:00Z)
• Meerkat Behaviour Recognition Dataset [3.53348643468069]
  We introduce a large meerkat behaviour recognition video dataset with diverse annotated behaviours. This dataset includes videos from two positions within the meerkat enclosure at the Wellington Zoo (Wellington, New Zealand)
  arXiv  Detail & Related papers  (2023-06-20T06:50:50Z)
• MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understanding [38.3767550066302]
  MammalNet is a large-scale animal behavior dataset with taxonomy-guided annotations of mammals and their common behaviors. It contains over 18K videos totaling 539 hours, which is 10 times larger than the largest existing animal behavior dataset. We establish three benchmarks on MammalNet: standard animal and behavior recognition, compositional low-shot animal and behavior recognition, and behavior detection.
  arXiv  Detail & Related papers  (2023-06-01T11:45:33Z)
• MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior [28.878568752724235]
  We introduce MABe22, a benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments. We test self-supervised video and trajectory representation learning methods to demonstrate the use of our benchmark.
  arXiv  Detail & Related papers  (2022-07-21T15:51:30Z)
• APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking [77.87449881852062]
  APT-36K is the first large-scale benchmark for animal pose estimation and tracking. It consists of 2,400 video clips collected and filtered from 30 animal species with 15 frames for each video, resulting in 36,000 frames in total. We benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking.
  arXiv  Detail & Related papers  (2022-06-12T07:18:36Z)
• AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild [51.35013619649463]
  We present an extensive dataset of free-running cheetahs in the wild, called AcinoSet. The dataset contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided.
  arXiv  Detail & Related papers  (2021-03-24T15:54:11Z)
• Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events [106.19047816743988]
  We present a new large-scale dataset with comprehensive annotations, named Human-in-Events or HiEve. It contains a record number of poses (>1M), the largest number of action instances (>56k) under complex events, as well as one of the largest numbers of trajectories lasting for longer time. Based on its diverse annotation, we present two simple baselines for action recognition and pose estimation.
  arXiv  Detail & Related papers  (2020-05-09T18:24:52Z)
• Comprehensive Instructional Video Analysis: The COIN Dataset and Performance Evaluation [100.68317848808327]
  We present a large-scale dataset named as "COIN" for COmprehensive INstructional video analysis. COIN dataset contains 11,827 videos of 180 tasks in 12 domains related to our daily life. With a new developed toolbox, all the videos are annotated efficiently with a series of step labels and the corresponding temporal boundaries.
  arXiv  Detail & Related papers  (2020-03-20T16:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.