Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary

• URL: http://arxiv.org/abs/2212.00266v1
• Date: Thu, 1 Dec 2022 04:23:18 GMT
• Title: Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary
• Authors: Shiting Xiao, Yufu Wang, Ammon Perkes, Bernd Pfrommer, Marc Schmidt, Kostas Daniilidis and Marc Badger
• Abstract summary: We introduce a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers.
• Score: 32.19504891200443
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to capture detailed interactions among individuals in a social group is foundational to our study of animal behavior and neuroscience. Recent advances in deep learning and computer vision are driving rapid progress in methods that can record the actions and interactions of multiple individuals simultaneously. Many social species, such as birds, however, live deeply embedded in a three-dimensional world. This world introduces additional perceptual challenges such as occlusions, orientation-dependent appearance, large variation in apparent size, and poor sensor coverage for 3D reconstruction, that are not encountered by applications studying animals that move and interact only on 2D planes. Here we introduce a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers. Finally, we analyze captured ethogram data and demonstrate that social context affects the distribution of sequential interactions between birds in the aviary.

Related papers

• EgoPet: Egomotion and Interaction Data from an Animal's Perspective [82.7192364237065]
  We introduce a dataset of pet egomotion imagery with diverse examples of simultaneous egomotion and multi-agent interaction. EgoPet offers a radically distinct perspective from existing egocentric datasets of humans or vehicles. We define two in-domain benchmark tasks that capture animal behavior, and a third benchmark to assess the utility of EgoPet as a pretraining resource to robotic quadruped locomotion.
  arXiv  Detail & Related papers  (2024-04-15T17:59:47Z)
• Computer Vision for Primate Behavior Analysis in the Wild [60.937374400311256]
  Video-based behavioral monitoring has great potential for transforming how we study animal cognition and behavior. There is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today. We start with a survey of the state-of-the-art methods for computer vision problems that are directly relevant to the video-based study of animal behavior. We then review methods for effort-efficient learning, which is one of the biggest challenges from a practical perspective.
  arXiv  Detail & Related papers  (2024-01-29T18:59:56Z)
• Behaviour Modelling of Social Animals via Causal Structure Discovery and Graph Neural Networks [15.542220566525021]
  We propose a method to build behavioural models using causal structure discovery and graph neural networks for time series. We apply this method to a mob of meerkats in a zoo environment and study its ability to predict future actions.
  arXiv  Detail & Related papers  (2023-12-21T23:34:08Z)
• Meerkat Behaviour Recognition Dataset [3.53348643468069]
  We introduce a large meerkat behaviour recognition video dataset with diverse annotated behaviours. This dataset includes videos from two positions within the meerkat enclosure at the Wellington Zoo (Wellington, New Zealand)
  arXiv  Detail & Related papers  (2023-06-20T06:50:50Z)
• Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey [6.555250822345809]
  In nature, the collective behavior of animals is dominated by the interactions between individuals of the same species. Recent advances in 3D vision have led to a number of impressive works on the 3D shape and pose estimation. This work is the first attempt to provide an overview of recent advances in 3D bird reconstruction based on monocular vision.
  arXiv  Detail & Related papers  (2022-07-10T18:13:25Z)
• HSPACE: Synthetic Parametric Humans Animated in Complex Environments [67.8628917474705]
  We build a large-scale photo-realistic dataset, Human-SPACE, of animated humans placed in complex indoor and outdoor environments. We combine a hundred diverse individuals of varying ages, gender, proportions, and ethnicity, with hundreds of motions and scenes, in order to generate an initial dataset of over 1 million frames. Assets are generated automatically, at scale, and are compatible with existing real time rendering and game engines.
  arXiv  Detail & Related papers  (2021-12-23T22:27:55Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• Beyond Tracking: Using Deep Learning to Discover Novel Interactions in Biological Swarms [3.441021278275805]
  We propose training deep network models to predict system-level states directly from generic graphical features from the entire view. Because the resulting predictive models are not based on human-understood predictors, we use explanatory modules. This represents an example of augmented intelligence in behavioral ecology -- knowledge co-creation in a human-AI team.
  arXiv  Detail & Related papers  (2021-08-20T22:50:41Z)
• Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model [124.26611454540813]
  Social behaviour analysis of mice is an invaluable tool to assess therapeutic efficacy of neurodegenerative diseases. Because of the potential to create rich descriptions of mouse social behaviors, the use of multi-view video recordings for rodent observations is increasingly receiving much attention. We propose a novel multiview latent-attention and dynamic discriminative model that jointly learns view-specific and view-shared sub-structures.
  arXiv  Detail & Related papers  (2020-11-04T18:09:58Z)
• 3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View [35.61330221535231]
  We introduce a model and multi-view optimization approach to capture the unique shape and pose space displayed by live birds. We then introduce a pipeline and experiments for keypoint, mask, pose, and shape regression that recovers accurate avian postures from single views. We provide extensive multi-view keypoint and mask annotations collected from a group of 15 social birds housed together in an outdoor aviary.
  arXiv  Detail & Related papers  (2020-08-13T23:29:04Z)
• Learning Human-Object Interaction Detection using Interaction Points [140.0200950601552]
  We propose a novel fully-convolutional approach that directly detects the interactions between human-object pairs. Our network predicts interaction points, which directly localize and classify the inter-action. Experiments are performed on two popular benchmarks: V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2020-03-31T08:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.