Dual-stream spatiotemporal networks with feature sharing for monitoring animals in the home cage

• URL: http://arxiv.org/abs/2206.00614v1
• Date: Wed, 1 Jun 2022 16:32:25 GMT
• Title: Dual-stream spatiotemporal networks with feature sharing for monitoring animals in the home cage
• Authors: Ezechukwu I. Nwokedi, Rasneer S. Bains, Luc Bidaut, Xujiong Ye, Sara Wells, James M. Brown
• Abstract summary: We introduce a feature-sharing approach that jointly the streams at regular intervals throughout the network. We achieve a prediction accuracy of 86.47% using an ensemble of Inception-based networks. Future work will investigate the effectiveness of sharing in behavioural classification in the unsupervised anomaly detection domain.
• Score: 0.9937939233206224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a spatiotemporal deep learning approach for mouse behavioural classification in the home cage. Using a series of dual-stream architectures with assorted modifications to increase performance, we introduce a novel feature-sharing approach that jointly processes the streams at regular intervals throughout the network. Using a publicly available labelled dataset of singly-housed mice, we achieve a prediction accuracy of 86.47% using an ensemble of Inception-based networks that utilize feature sharing. We also demonstrate through ablation studies that for all models, the feature-sharing architectures consistently perform better than conventional ones having separate streams. The best performing models were further evaluated on other activity datasets, both mouse and human, and achieved state-of-the-art results. Future work will investigate the effectiveness of feature sharing in behavioural classification in the unsupervised anomaly detection domain.

Related papers

• Efficient Mixture of Geographical Species for On Device Wildlife Monitoring [2.8718221966298754]
  In this work, we explore the training of a single species detector which uses conditional to bias structured sub networks in a geographically-aware manner. We propose a method for pruning the expert model per location and demonstrate conditional computation performance on two geographically distributed datasets: iNaturalist and iWildcam.
  arXiv  Detail & Related papers  (2025-04-11T15:25:36Z)
• Local Consensus Enhanced Siamese Network with Reciprocal Loss for Two-view Correspondence Learning [35.5851523517487]
  Two-view correspondence learning usually establish an end-to-end network to jointly predict correspondence reliability and relative pose. We propose a Local Feature Consensus (LFC) plugin block to augment the features of existing models. We extend existing models to a Siamese network with a reciprocal loss that exploits the supervision of mutual projection.
  arXiv  Detail & Related papers  (2023-08-06T22:20:09Z)
• Regularization Through Simultaneous Learning: A Case Study on Plant Classification [0.0]
  This paper introduces Simultaneous Learning, a regularization approach drawing on principles of Transfer Learning and Multi-task Learning. We leverage auxiliary datasets with the target dataset, the UFOP-HVD, to facilitate simultaneous classification guided by a customized loss function. Remarkably, our approach demonstrates superior performance over models without regularization.
  arXiv  Detail & Related papers  (2023-05-22T19:44:57Z)
• Improving Out-of-Distribution Generalization of Neural Rerankers with Contextualized Late Interaction [52.63663547523033]
  Late interaction, the simplest form of multi-vector, is also helpful to neural rerankers that only use the [] vector to compute the similarity score. We show that the finding is consistent across different model sizes and first-stage retrievers of diverse natures.
  arXiv  Detail & Related papers  (2023-02-13T18:42:17Z)
• Real-Time Cattle Interaction Recognition via Triple-stream Network [3.3843451892622576]
  Cattle localization network outputs high-quality interaction proposals from every detected cattle. Interaction recognition network feeds them into the interaction recognition network with a triple-stream architecture.
  arXiv  Detail & Related papers  (2022-09-06T06:31:09Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Beyond Transfer Learning: Co-finetuning for Action Localisation [64.07196901012153]
  We propose co-finetuning -- simultaneously training a single model on multiple upstream'' and downstream'' tasks. We demonstrate that co-finetuning outperforms traditional transfer learning when using the same total amount of data. We also show how we can easily extend our approach to multiple upstream'' datasets to further improve performance.
  arXiv  Detail & Related papers  (2022-07-08T10:25:47Z)
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach [80.8446673089281]
  We propose a new learning paradigm with graph representation and learning. Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
  arXiv  Detail & Related papers  (2021-10-09T09:02:45Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction [15.270499225813841]
  We propose a 3D fully convolutional architecture for video saliency prediction. We provide the base hierarchical learning mechanism with two techniques for domain adaptation and domain-specific learning. The results of our experiments show that the proposed model yields state-of-the-art accuracy on supervised saliency prediction.
  arXiv  Detail & Related papers  (2020-10-02T23:00:00Z)
• Adversarial Bipartite Graph Learning for Video Domain Adaptation [50.68420708387015]
  Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area. Recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations are not highly effective on the videos. This paper proposes an Adversarial Bipartite Graph (ABG) learning framework which directly models the source-target interactions.
  arXiv  Detail & Related papers  (2020-07-31T03:48:41Z)
• A Neural Architecture for Detecting Confusion in Eye-tracking Data [1.8655840060559168]
  We introduce an architecture that uses RNN and CNN sub-models in parallel to take advantage of the temporal and visuospatial aspects of our data. Our model outperforms an existing model based on Random Forests resulting in a 22% improvement in combined sensitivity & specificity.
  arXiv  Detail & Related papers  (2020-03-13T18:20:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.