Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model
- URL: http://arxiv.org/abs/2011.02451v2
- Date: Wed, 30 Jun 2021 07:44:00 GMT
- Title: Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model
- Authors: Zheheng Jiang, Feixiang Zhou, Aite Zhao, Xin Li, Ling Li, Dacheng Tao,
Xuelong Li and Huiyu Zhou
- Abstract summary: Social behaviour analysis of mice is an invaluable tool to assess therapeutic efficacy of neurodegenerative diseases.
Because of the potential to create rich descriptions of mouse social behaviors, the use of multi-view video recordings for rodent observations is increasingly receiving much attention.
We propose a novel multiview latent-attention and dynamic discriminative model that jointly learns view-specific and view-shared sub-structures.
- Score: 124.26611454540813
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Home-cage social behaviour analysis of mice is an invaluable tool to assess
therapeutic efficacy of neurodegenerative diseases. Despite tremendous efforts
made within the research community, single-camera video recordings are mainly
used for such analysis. Because of the potential to create rich descriptions of
mouse social behaviors, the use of multi-view video recordings for rodent
observations is increasingly receiving much attention. However, identifying
social behaviours from various views is still challenging due to the lack of
correspondence across data sources. To address this problem, we here propose a
novel multiview latent-attention and dynamic discriminative model that jointly
learns view-specific and view-shared sub-structures, where the former captures
unique dynamics of each view whilst the latter encodes the interaction between
the views. Furthermore, a novel multi-view latent-attention variational
autoencoder model is introduced in learning the acquired features, enabling us
to learn discriminative features in each view. Experimental results on the
standard CRMI13 and our multi-view Parkinson's Disease Mouse Behaviour (PDMB)
datasets demonstrate that our model outperforms the other state of the arts
technologies and effectively deals with the imbalanced data problem.
Related papers
- Neuromorphic Face Analysis: a Survey [26.357357272526322]
Neuromorphic sensors, also known as event cameras, are a class of imaging devices mimicking the function of biological visual systems.
These properties have proven to be interesting in modeling human faces, both from an effectiveness and a privacy-preserving point of view.
This survey paper presents a comprehensive overview of capabilities, challenges and emerging applications in the domain of neuromorphic face analysis.
arXiv Detail & Related papers (2024-02-18T16:17:25Z) - Generating Human-Centric Visual Cues for Human-Object Interaction
Detection via Large Vision-Language Models [59.611697856666304]
Human-object interaction (HOI) detection aims at detecting human-object pairs and predicting their interactions.
We propose three prompts with VLM to generate human-centric visual cues within an image from multiple perspectives of humans.
We develop a transformer-based multimodal fusion module with multitower architecture to integrate visual cue features into the instance and interaction decoders.
arXiv Detail & Related papers (2023-11-26T09:11:32Z) - Cross-Skeleton Interaction Graph Aggregation Network for Representation
Learning of Mouse Social Behaviour [24.716092330419123]
Social behaviour analysis of mice has become an increasingly popular research area in behavioural neuroscience.
It is challenging to model complex social interactions between mice due to highly deformable body shapes and ambiguous movement patterns.
We propose a Cross-Skeleton Interaction Graph Aggregation Network (CS-IGANet) to learn abundant dynamics of freely interacting mice.
arXiv Detail & Related papers (2022-08-07T21:06:42Z) - Incorporating Heterogeneous User Behaviors and Social Influences for
Predictive Analysis [32.31161268928372]
We aim to incorporate heterogeneous user behaviors and social influences for behavior predictions.
This paper proposes a variant of Long-Short Term Memory (LSTM) which can consider context while a behavior sequence.
A residual learning-based decoder is designed to automatically construct multiple high-order cross features based on social behavior representation.
arXiv Detail & Related papers (2022-07-24T17:05:37Z) - Analyzing the Effects of Handling Data Imbalance on Learned Features
from Medical Images by Looking Into the Models [50.537859423741644]
Training a model on an imbalanced dataset can introduce unique challenges to the learning problem.
We look deeper into the internal units of neural networks to observe how handling data imbalance affects the learned features.
arXiv Detail & Related papers (2022-04-04T09:38:38Z) - Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
3D pose data can now be reliably extracted from multi-view video sequences without manual intervention.
We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations.
To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
arXiv Detail & Related papers (2021-12-02T12:45:46Z) - Overcoming the Domain Gap in Contrastive Learning of Neural Action
Representations [60.47807856873544]
A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior.
We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies.
This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
arXiv Detail & Related papers (2021-11-29T15:27:51Z) - Beyond Tracking: Using Deep Learning to Discover Novel Interactions in
Biological Swarms [3.441021278275805]
We propose training deep network models to predict system-level states directly from generic graphical features from the entire view.
Because the resulting predictive models are not based on human-understood predictors, we use explanatory modules.
This represents an example of augmented intelligence in behavioral ecology -- knowledge co-creation in a human-AI team.
arXiv Detail & Related papers (2021-08-20T22:50:41Z) - Modeling Shared Responses in Neuroimaging Studies through MultiView ICA [94.31804763196116]
Group studies involving large cohorts of subjects are important to draw general conclusions about brain functional organization.
We propose a novel MultiView Independent Component Analysis model for group studies, where data from each subject are modeled as a linear combination of shared independent sources plus noise.
We demonstrate the usefulness of our approach first on fMRI data, where our model demonstrates improved sensitivity in identifying common sources among subjects.
arXiv Detail & Related papers (2020-06-11T17:29:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.