Context-Aware Network Based on Multi-scale Spatio-temporal Attention for Action Recognition in Videos

• URL: http://arxiv.org/abs/2512.18750v1
• Date: Sun, 21 Dec 2025 14:34:49 GMT
• Title: Context-Aware Network Based on Multi-scale Spatio-temporal Attention for Action Recognition in Videos
• Authors: Xiaoyang Li, Wenzhu Yang, Kanglin Wang, Tiebiao Wang, Qingsong Fei,
• Abstract summary: We introduce the Context-Aware Network (CAN)<n>CAN consists of two core modules: the Multi-scale Temporal Cue Module (MTCM) and the Group Spatial Cue Module (GSCM)<n>Our approach achieves competitive performance, outperforming most mainstream methods, with accuracies of 50.4% on Something-Something V1, 63.9% on Something-Something2, 88.4% on Diving48, 74.9% on Kinetics-400, and 86.9% on UCF101.
• Score: 2.729217186919621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Action recognition is a critical task in video understanding, requiring the comprehensive capture of spatio-temporal cues across various scales. However, existing methods often overlook the multi-granularity nature of actions. To address this limitation, we introduce the Context-Aware Network (CAN). CAN consists of two core modules: the Multi-scale Temporal Cue Module (MTCM) and the Group Spatial Cue Module (GSCM). MTCM effectively extracts temporal cues at multiple scales, capturing both fast-changing motion details and overall action flow. GSCM, on the other hand, extracts spatial cues at different scales by grouping feature maps and applying specialized extraction methods to each group. Experiments conducted on five benchmark datasets (Something-Something V1 and V2, Diving48, Kinetics-400, and UCF101) demonstrate the effectiveness of CAN. Our approach achieves competitive performance, outperforming most mainstream methods, with accuracies of 50.4% on Something-Something V1, 63.9% on Something-Something V2, 88.4% on Diving48, 74.9% on Kinetics-400, and 86.9% on UCF101. These results highlight the importance of capturing multi-scale spatio-temporal cues for robust action recognition.

Related papers

• PRISM: Performer RS-IMLE for Single-pass Multisensory Imitation Learning [51.24484551729328]
  We introduce PRISM, a single-pass policy based on a batch-global rejection-sampling variant of IMLE.<n> PRISM couples a temporal multisensory encoder with a linear-attention generator using a Performer architecture.<n>We demonstrate the efficacy of PRISM on a diverse real-world hardware suite, including loco-manipulation using a Unitree Go2 with a 7-DoF arm D1 and tabletop manipulation with a UR5 manipulator.
  arXiv  Detail & Related papers  (2026-02-02T17:57:37Z)
• ActionAtlas: A VideoQA Benchmark for Domain-specialized Action Recognition [111.32822459456793]
  ActionAtlas is a video question answering benchmark featuring short videos across various sports. The dataset includes 934 videos showcasing 580 unique actions across 56 sports, with a total of 1896 actions within choices. We evaluate open and proprietary foundation models on this benchmark, finding that the best model, GPT-4o, achieves a maximum accuracy of 45.52%.
  arXiv  Detail & Related papers  (2024-10-08T07:55:09Z)
• DIR-AS: Decoupling Individual Identification and Temporal Reasoning for Action Segmentation [84.78383981697377]
  Fully supervised action segmentation works on frame-wise action recognition with dense annotations and often suffers from the over-segmentation issue. We develop a novel local-global attention mechanism with temporal pyramid dilation and temporal pyramid pooling for efficient multi-scale attention. We achieve state-of-the-art accuracy, eg, 82.8% (+2.6%) on GTEA and 74.7% (+1.2%) on Breakfast, which demonstrates the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-04-04T20:27:18Z)
• GaitMM: Multi-Granularity Motion Sequence Learning for Gait Recognition [6.877671230651998]
  Gait recognition aims to identify individual-specific walking patterns by observing the different periodic movements of each body part. Most existing methods treat each part equally and fail to account for the data redundancy caused by the different step frequencies and sampling rates of gait. In this study, we propose a multi-granularity motion representation (GaitMM) for gait sequence learning.
  arXiv  Detail & Related papers  (2022-09-18T04:07:33Z)
• Slow-Fast Visual Tempo Learning for Video-based Action Recognition [78.3820439082979]
  Action visual tempo characterizes the dynamics and the temporal scale of an action. Previous methods capture the visual tempo either by sampling raw videos with multiple rates, or by hierarchically sampling backbone features. We propose a Temporal Correlation Module (TCM) to extract action visual tempo from low-level backbone features at single-layer remarkably.
  arXiv  Detail & Related papers  (2022-02-24T14:20:04Z)
• Learning from Temporal Gradient for Semi-supervised Action Recognition [15.45239134477737]
  We introduce temporal gradient as an additional modality for more attentive feature extraction. Our method achieves the state-of-the-art performance on three video action recognition benchmarks.
  arXiv  Detail & Related papers  (2021-11-25T20:30:30Z)
• Skeleton-Split Framework using Spatial Temporal Graph Convolutional Networks for Action Recogntion [2.132096006921048]
  This work aims to recognize activities of daily living using the ST-GCN model. We have achieved 48.88 % top-1 accuracy by using the connection split partitioning approach. accuracy of 73.25 % top-1 is achieved by using the index split partitioning strategy.
  arXiv  Detail & Related papers  (2021-11-04T18:59:02Z)
• TSI: Temporal Saliency Integration for Video Action Recognition [32.18535820790586]
  We propose a Temporal Saliency Integration (TSI) block, which mainly contains a Salient Motion Excitation (SME) module and a Cross-scale Temporal Integration (CTI) module. SME aims to highlight the motion-sensitive area through local-global motion modeling. CTI is designed to perform multi-scale temporal modeling through a group of separate 1D convolutions respectively.
  arXiv  Detail & Related papers  (2021-06-02T11:43:49Z)
• Learning Salient Boundary Feature for Anchor-free Temporal Action Localization [81.55295042558409]
  Temporal action localization is an important yet challenging task in video understanding. We propose the first purely anchor-free temporal localization method. Our model includes (i) an end-to-end trainable basic predictor, (ii) a saliency-based refinement module, and (iii) several consistency constraints.
  arXiv  Detail & Related papers  (2021-03-24T12:28:32Z)
• Learning Comprehensive Motion Representation for Action Recognition [124.65403098534266]
  2D CNN-based methods are efficient but may yield redundant features due to applying the same 2D convolution kernel to each frame. Recent efforts attempt to capture motion information by establishing inter-frame connections while still suffering the limited temporal receptive field or high latency. We propose a Channel-wise Motion Enhancement (CME) module to adaptively emphasize the channels related to dynamic information with a channel-wise gate vector. We also propose a Spatial-wise Motion Enhancement (SME) module to focus on the regions with the critical target in motion, according to the point-to-point similarity between adjacent feature maps.
  arXiv  Detail & Related papers  (2021-03-23T03:06:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.