APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking

• URL: http://arxiv.org/abs/2206.05683v1
• Date: Sun, 12 Jun 2022 07:18:36 GMT
• Title: APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking
• Authors: Yuxiang Yang, Junjie Yang, Yufei Xu, Jing Zhang, Long Lan, Dacheng Tao
• Abstract summary: APT-36K is the first large-scale benchmark for animal pose estimation and tracking. It consists of 2,400 video clips collected and filtered from 30 animal species with 15 frames for each video, resulting in 36,000 frames in total. We benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking.
• Score: 77.87449881852062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Animal pose estimation and tracking (APT) is a fundamental task for detecting and tracking animal keypoints from a sequence of video frames. Previous animal-related datasets focus either on animal tracking or single-frame animal pose estimation, and never on both aspects. The lack of APT datasets hinders the development and evaluation of video-based animal pose estimation and tracking methods, limiting real-world applications, e.g., understanding animal behavior in wildlife conservation. To fill this gap, we make the first step and propose APT-36K, i.e., the first large-scale benchmark for animal pose estimation and tracking. Specifically, APT-36K consists of 2,400 video clips collected and filtered from 30 animal species with 15 frames for each video, resulting in 36,000 frames in total. After manual annotation and careful double-check, high-quality keypoint and tracking annotations are provided for all the animal instances. Based on APT-36K, we benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking. Based on the experimental results, we gain some empirical insights and show that APT-36K provides a valuable animal pose estimation and tracking benchmark, offering new challenges and opportunities for future research. The code and dataset will be made publicly available at https://github.com/pandorgan/APT-36K.

Related papers

• Towards Multi-Modal Animal Pose Estimation: An In-Depth Analysis [48.57353513938747]
  Animal pose estimation (APE) aims to locate the animal body parts using a diverse array of sensor and modality inputs. By evaluating 178 papers since 2013, APE methods are categorised by sensor and modality types, learning paradigms, experimental setup, and application domains.
  arXiv  Detail & Related papers  (2024-10-12T00:37:07Z)
• PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptions [57.871692507044344]
  Pose estimation aims to accurately identify anatomical keypoints in humans and animals using monocular images. Current models are typically trained and tested on clean data, potentially overlooking the corruption during real-world deployment. We introduce PoseBench, a benchmark designed to evaluate the robustness of pose estimation models against real-world corruption.
  arXiv  Detail & Related papers  (2024-06-20T14:40:17Z)
• OpenAnimalTracks: A Dataset for Animal Track Recognition [2.3020018305241337]
  We introduce OpenAnimalTracks dataset, the first publicly available labeled dataset designed to facilitate the automated classification and detection of animal footprints. We show the potential of automated footprint identification with representative classifiers and detection models. We hope our dataset paves the way for automated animal tracking techniques, enhancing our ability to protect and manage biodiversity.
  arXiv  Detail & Related papers  (2024-06-14T00:37:17Z)
• APTv2: Benchmarking Animal Pose Estimation and Tracking with a Large-scale Dataset and Beyond [27.50166679588048]
  APTv2 is the pioneering large-scale benchmark for animal pose estimation and tracking. It comprises 2,749 video clips filtered and collected from 30 distinct animal species. We provide high-quality keypoint and tracking annotations for a total of 84,611 animal instances.
  arXiv  Detail & Related papers  (2023-12-25T04:49:49Z)
• OmniMotionGPT: Animal Motion Generation with Limited Data [70.35662376853163]
  We introduce AnimalML3D, the first text-animal motion dataset with 1240 animation sequences spanning 36 different animal identities. We are able to generate animal motions with high diversity and fidelity, quantitatively and qualitatively outperforming the results of training human motion generation baselines on animal data.
  arXiv  Detail & Related papers  (2023-11-30T07:14:00Z)
• AnimalTrack: A Large-scale Benchmark for Multi-Animal Tracking in the Wild [26.794672185860538]
  We introduce AnimalTrack, a large-scale benchmark for multi-animal tracking in the wild. AnimalTrack consists of 58 sequences from a diverse selection of 10 common animal categories. We extensively evaluate 14 state-of-the-art representative trackers.
  arXiv  Detail & Related papers  (2022-04-30T04:23:59Z)
• Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding [4.606145900630665]
  We create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks. Our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments. We propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals.
  arXiv  Detail & Related papers  (2022-04-18T02:05:15Z)
• AP-10K: A Benchmark for Animal Pose Estimation in the Wild [83.17759850662826]
  We propose AP-10K, the first large-scale benchmark for general animal pose estimation. AP-10K consists of 10,015 images collected and filtered from 23 animal families and 60 species. Results provide sound empirical evidence on the superiority of learning from diverse animals species in terms of both accuracy and generalization ability.
  arXiv  Detail & Related papers  (2021-08-28T10:23:34Z)
• AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild [51.35013619649463]
  We present an extensive dataset of free-running cheetahs in the wild, called AcinoSet. The dataset contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided.
  arXiv  Detail & Related papers  (2021-03-24T15:54:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.