MSGL-Transformer: A Multi-Scale Global-Local Transformer for Rodent Social Behavior Recognition

• URL: http://arxiv.org/abs/2604.07578v1
• Date: Wed, 08 Apr 2026 20:34:47 GMT
• Title: MSGL-Transformer: A Multi-Scale Global-Local Transformer for Rodent Social Behavior Recognition
• Abstract summary: MSGL-Transformer is a Global-Local Transformer for recognizing rodent social behaviors from pose-based temporal sequences.<n> architecture integrates parallel short-range, medium-range, and global attention branches to capture behavior dynamics.<n>On RatSI, MSGL-Transformer achieves 75.4% mean accuracy and F1-score of 0.745 across nine cross-validation splits, outperforming TCN, LSTM, and Bi-LSTM.<n>On CalMS21, it achieves 87.1% accuracy and F1-score of 0.8745, a +10.7% improvement over HSTWFormer, and outperforms ST-GCN, MS-G
• Score: 3.8958737942482204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recognition of rodent behavior is important for understanding neural and behavioral mechanisms. Traditional manual scoring is time-consuming and prone to human error. We propose MSGL-Transformer, a Multi-Scale Global-Local Transformer for recognizing rodent social behaviors from pose-based temporal sequences. The model employs a lightweight transformer encoder with multi-scale attention to capture motion dynamics across different temporal scales. The architecture integrates parallel short-range, medium-range, and global attention branches to explicitly capture behavior dynamics at multiple temporal scales. We also introduce a Behavior-Aware Modulation (BAM) block, inspired by SE-Networks, which modulates temporal embeddings to emphasize behavior-relevant features prior to attention. We evaluate on two datasets: RatSI (5 behavior classes, 12D pose inputs) and CalMS21 (4 behavior classes, 28D pose inputs). On RatSI, MSGL-Transformer achieves 75.4% mean accuracy and F1-score of 0.745 across nine cross-validation splits, outperforming TCN, LSTM, and Bi-LSTM. On CalMS21, it achieves 87.1% accuracy and F1-score of 0.8745, a +10.7% improvement over HSTWFormer, and outperforms ST-GCN, MS-G3D, CTR-GCN, and STGAT. The same architecture generalizes across both datasets with only input dimensionality and number of classes adjusted.

Related papers

• Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies [1.091319386322552]
  This study addresses two challenges in automated detection of autism-related self-stimulatory behaviors from video.<n>It identifies the optimal sequence-based neural network architecture and temporal sampling rate.<n>It also characterizes data augmentation strategies for training on small behavioral datasets.
  arXiv  Detail & Related papers  (2026-07-08T22:22:37Z)
• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• Exploitation of Hidden Context in Dynamic Movement Forecasting: A Neural Network Journey from Recurrent to Graph Neural Networks and General Purpose Transformers [2.297609455004023]
  Traditional approaches, including Kalman filters, often struggle to model the non-linear dynamics present in such scenarios.<n>Machine learning methods, such as long short-term memory (LSTM) networks, graph neural networks (GNNs), and Transformers, offer greater flexibility and accuracy.<n>In this paper, we evaluate these models and assess their strengths and weaknesses.
  arXiv  Detail & Related papers  (2026-05-14T14:02:45Z)
• Transformer-Based Wildlife Species Classification from Daily Movement Trajectories [13.570863502498336]
  We train sequence models on large-scale, 7-species GPS trajectories.<n>Trajectories models are evaluated using a protocol in which entire telemetry studies or regions are heldout during testing.<n>Transformers consistently achieve higher balanced accuracy with gains of approximately 8 to 22 percentage points.
  arXiv  Detail & Related papers  (2026-05-07T11:20:55Z)
• Beyond Sequences: A Benchmark for Atomic Hand-Object Interaction Using a Static RNN Encoder [0.0]
  We introduce a structured data engineering process that converts raw videos from the MANIAC dataset into 27,476 statistical-kinematic feature vectors.<n>Our model successfully overcame the most challenging transitional class, 'grabbing', by achieving a balanced F1-score of 0.90.<n>These findings provide a new benchmark for low-level hand-object interaction recognition using structured, interpretable features and lightweight architectures.
  arXiv  Detail & Related papers  (2025-12-10T13:11:43Z)
• Gated Fusion Enhanced Multi-Scale Hierarchical Graph Convolutional Network for Stock Movement Prediction [12.806794559293289]
  We introduce MS-HGFN (Multi-Scale Hierarchical Graph Fusion Network)<n>The model features a hierarchical GNN module that forms dynamic graphs by learning patterns from intra-attributes and features from inter-attributes over different time scales.<n> Experiments utilizing real-world datasets from U.S. and Chinese stock markets demonstrate that MS-HGFN outperforms both traditional and advanced models.
  arXiv  Detail & Related papers  (2025-11-03T13:39:23Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• SETransformer: A Hybrid Attention-Based Architecture for Robust Human Activity Recognition [7.291558599547268]
  Human Activity Recognition (HAR) using wearable sensor data has become a central task in mobile computing, healthcare, and human-computer interaction.<n>We propose SETransformer, a hybrid deep neural architecture that combines Transformer-based temporal modeling with channel-wise squeeze-and-excitation (SE) attention and a learnable temporal attention pooling mechanism.<n>We evaluate SETransformer on the WISDM dataset and demonstrate that it significantly outperforms conventional models including LSTM, GRU, BiLSTM, and CNN baselines.
  arXiv  Detail & Related papers  (2025-05-25T23:39:34Z)
• Ensemble-Enhanced Graph Autoencoder with GAT and Transformer-Based Encoders for Robust Fault Diagnosis [0.0]
  This study introduces a novel graph-based framework for fault classification.<n>It converts time-series vibration data from machinery operating at varying horsepower levels into a graph representation.<n>A Graph Auto (GAE) with a deep graph transformer encoder, decoder, and ensemble is developed to learn latent graph representations and classify faults.
  arXiv  Detail & Related papers  (2025-04-13T04:21:11Z)
• Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation [53.04781510348416]
  Video-based 3D human pose and shape estimations are evaluated by intra-frame accuracy and inter-frame smoothness. We propose to structurally decouple the modeling of long-term and short-term correlations in an end-to-end framework, Global-to-Local Transformer (GLoT) Our GLoT surpasses previous state-of-the-art methods with the lowest model parameters on popular benchmarks, i.e., 3DPW, MPI-INF-3DHP, and Human3.6M.
  arXiv  Detail & Related papers  (2023-03-26T14:57:49Z)
• A Close Look at Spatial Modeling: From Attention to Convolution [70.5571582194057]
  Vision Transformers have shown great promise recently for many vision tasks due to the insightful architecture design and attention mechanism. We generalize self-attention formulation to abstract a queryirrelevant global context directly and integrate the global context into convolutions. With less than 14M parameters, our FCViT-S12 outperforms related work ResT-Lite by 3.7% top1 accuracy on ImageNet-1K.
  arXiv  Detail & Related papers  (2022-12-23T19:13:43Z)
• Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking [59.79252390626194]
  We propose a novel solution named TransSTAM, which leverages Transformer to model both the appearance features of each object and the spatial-temporal relationships among objects. The proposed method is evaluated on multiple public benchmarks including MOT16, MOT17, and MOT20, and it achieves a clear performance improvement in both IDF1 and HOTA.
  arXiv  Detail & Related papers  (2022-05-31T01:19:18Z)
• Uniformer: Unified Transformer for Efficient Spatiotemporal Representation Learning [68.55487598401788]
  Recent advances in this research have been mainly driven by 3D convolutional neural networks and vision transformers. We propose a novel Unified transFormer (UniFormer) which seamlessly integrates merits of 3D convolution self-attention in a concise transformer format. We conduct extensive experiments on the popular video benchmarks, e.g., Kinetics-400, Kinetics-600, and Something-Something V1&V2. Our UniFormer achieves 8/84.8% top-1 accuracy on Kinetics-400/Kinetics-600, while requiring 10x fewer GFLOPs than other state-of-the-art methods
  arXiv  Detail & Related papers  (2022-01-12T20:02:32Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.