Swarm behavior tracking based on a deep vision algorithm
- URL: http://arxiv.org/abs/2204.03319v1
- Date: Thu, 7 Apr 2022 09:32:12 GMT
- Title: Swarm behavior tracking based on a deep vision algorithm
- Authors: Meihong Wu, Xiaoyan Cao, Shihui Guo
- Abstract summary: We propose a detection and tracking framework for multi-ant tracking in the videos.
Our method runs 6-10 times faster than existing methods for insect tracking.
- Score: 5.070542698701158
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The intelligent swarm behavior of social insects (such as ants) springs up in
different environments, promising to provide insights for the study of embodied
intelligence. Researching swarm behavior requires that researchers could
accurately track each individual over time. Obviously, manually labeling
individual insects in a video is labor-intensive. Automatic tracking methods,
however, also poses serious challenges: (1) individuals are small and similar
in appearance; (2) frequent interactions with each other cause severe and
long-term occlusion. With the advances of artificial intelligence and computing
vision technologies, we are hopeful to provide a tool to automate monitor
multiple insects to address the above challenges. In this paper, we propose a
detection and tracking framework for multi-ant tracking in the videos by: (1)
adopting a two-stage object detection framework using ResNet-50 as backbone and
coding the position of regions of interest to locate ants accurately; (2) using
the ResNet model to develop the appearance descriptors of ants; (3)
constructing long-term appearance sequences and combining them with motion
information to achieve online tracking. To validate our method, we construct an
ant database including 10 videos of ants from different indoor and outdoor
scenes. We achieve a state-of-the-art performance of 95.7\% mMOTA and 81.1\%
mMOTP in indoor videos, 81.8\% mMOTA and 81.9\% mMOTP in outdoor videos.
Additionally, Our method runs 6-10 times faster than existing methods for
insect tracking. Experimental results demonstrate that our method provides a
powerful tool for accelerating the unraveling of the mechanisms underlying the
swarm behavior of social insects.
Related papers
- Computer Vision for Primate Behavior Analysis in the Wild [61.08941894580172]
Video-based behavioral monitoring has great potential for transforming how we study animal cognition and behavior.
There is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today.
arXiv Detail & Related papers (2024-01-29T18:59:56Z) - Motion Informed Object Detection of Small Insects in Time-lapse Camera
Recordings [1.3965477771846408]
We present a method pipeline for detecting insects in time-lapse RGB images.
Motion-Informed-Enhancement technique uses motion and colors to enhance insects in images.
The method improves the deep learning object detectors You Only Look Once (YOLO) and Faster Region-based CNN (Faster R-CNN)
arXiv Detail & Related papers (2022-12-01T10:54:06Z) - TempNet: Temporal Attention Towards the Detection of Animal Behaviour in
Videos [63.85815474157357]
We propose an efficient computer vision- and deep learning-based method for the detection of biological behaviours in videos.
TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder.
We demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events.
arXiv Detail & Related papers (2022-11-17T23:55:12Z) - Understandable Controller Extraction from Video Observations of Swarms [0.0]
Swarm behavior emerges from the local interaction of agents and their environment often encoded as simple rules.
We develop a method to automatically extract understandable swarm controllers from video demonstrations.
arXiv Detail & Related papers (2022-09-02T15:28:28Z) - MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations
of Behavior [28.878568752724235]
We introduce MABe22, a benchmark to assess the quality of learned behavior representations.
This dataset is collected from a variety of biology experiments.
We test self-supervised video and trajectory representation learning methods to demonstrate the use of our benchmark.
arXiv Detail & Related papers (2022-07-21T15:51:30Z) - A dataset of ant colonies motion trajectories in indoor and outdoor
scenes for social cluster behavior study [13.391307807956675]
In this paper, we collect 10 videos of ant colonies from different indoor and outdoor scenes.
In all 5354 frames, the location information and the identification number of each ant are recorded for a total of 712 ants and 114112 annotations.
It is hoped that this dataset will contribute to a deeper exploration on the behavior of the ant colony.
arXiv Detail & Related papers (2022-04-09T03:49:55Z) - Weakly Supervised Human-Object Interaction Detection in Video via
Contrastive Spatiotemporal Regions [81.88294320397826]
A system does not know what human-object interactions are present in a video as or the actual location of the human and object.
We introduce a dataset comprising over 6.5k videos with human-object interaction that have been curated from sentence captions.
We demonstrate improved performance over weakly supervised baselines adapted to our annotations on our video dataset.
arXiv Detail & Related papers (2021-10-07T15:30:18Z) - ASCNet: Self-supervised Video Representation Learning with
Appearance-Speed Consistency [62.38914747727636]
We study self-supervised video representation learning, which is a challenging task due to 1) a lack of labels for explicit supervision and 2) unstructured and noisy visual information.
Existing methods mainly use contrastive loss with video clips as the instances and learn visual representation by discriminating instances from each other.
In this paper, we observe that the consistency between positive samples is the key to learn robust video representations.
arXiv Detail & Related papers (2021-06-04T08:44:50Z) - Self-supervised Video Object Segmentation [76.83567326586162]
The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a.k.a. dense tracking)
We make the following contributions: (i) we propose to improve the existing self-supervised approach, with a simple, yet more effective memory mechanism for long-term correspondence matching; (ii) by augmenting the self-supervised approach with an online adaptation module, our method successfully alleviates tracker drifts caused by spatial-temporal discontinuity; (iv) we demonstrate state-of-the-art results among the self-supervised approaches on DAVIS-2017 and YouTube
arXiv Detail & Related papers (2020-06-22T17:55:59Z) - FLIVVER: Fly Lobula Inspired Visual Velocity Estimation & Ranging [0.0]
A tiny insect or insect-sized robot could estimate its absolute velocity and distance to nearby objects remains unknown.
We present a novel algorithm, FLIVVER, which combines the geometry of dynamic forward motion with inspiration from insect visual processing.
Our algorithm provides a clear hypothesis for how insects might estimate absolute velocity, and also provides a theoretical framework for designing fast analog circuitry for efficient state estimation.
arXiv Detail & Related papers (2020-04-10T22:35:13Z) - Human Motion Transfer from Poses in the Wild [61.6016458288803]
We tackle the problem of human motion transfer, where we synthesize novel motion video for a target person that imitates the movement from a reference video.
It is a video-to-video translation task in which the estimated poses are used to bridge two domains.
We introduce a novel pose-to-video translation framework for generating high-quality videos that are temporally coherent even for in-the-wild pose sequences unseen during training.
arXiv Detail & Related papers (2020-04-07T05:59:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.