STARK: Spatio-Temporal Attention for Representation of Keypoints for Continuous Sign Language Recognition

• URL: http://arxiv.org/abs/2603.16163v1
• Date: Tue, 17 Mar 2026 06:29:57 GMT
• Title: STARK: Spatio-Temporal Attention for Representation of Keypoints for Continuous Sign Language Recognition
• Abstract summary: This paper introduces a unified-temporal attention network that computes attention scores both spatially (across keypoints) and temporally (within local windows)<n>The encoder contains approximately $70-80$ fewer parameters than existing state-of-the-art models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continuous Sign Language Recognition (CSLR) is a crucial task for understanding the languages of deaf communities. Contemporary keypoint-based approaches typically rely on spatio-temporal encoding, where spatial interactions among keypoints are modeled using Graph Convolutional Networks or attention mechanisms, while temporal dynamics are captured using 1D convolutional networks. However, such designs often introduce a large number of parameters in both the encoder and the decoder. This paper introduces a unified spatio-temporal attention network that computes attention scores both spatially (across keypoints) and temporally (within local windows), and aggregates features to produce a local context-aware spatio-temporal representation. The proposed encoder contains approximately $70-80\%$ fewer parameters than existing state-of-the-art models while achieving comparable performance to keypoint-based methods on the Phoenix-14T dataset.

Related papers

• SigWavNet: Learning Multiresolution Signal Wavelet Network for Speech Emotion Recognition [17.568724398229232]
  Speech emotion recognition (SER) plays an important role in emotional states from deciphering speech signals.<n>This paper introduces a new end-to-end (E2E) deep learning multi-resolution framework for SER.<n>It exploits the capabilities of wavelets for effective localization in both time and frequency domains.
  arXiv  Detail & Related papers  (2025-02-01T04:18:06Z)
• Efficient Spatio-Temporal Signal Recognition on Edge Devices Using PointLCA-Net [0.0]
  This paper presents an approach that combines PointNet's feature extraction with the in-memory computing capabilities and energy efficiency of neuromorphic systems fortemporal signal recognition.<n>PointNet achieves high accuracy and significantly lower energy burden during both inference and training than comparable approaches.
  arXiv  Detail & Related papers  (2024-11-21T20:48:40Z)
• NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understanding [60.74434735079253]
  We propose a method known as Neighborhood Attention with Convolutions TCN (NAC-TCN) We accomplish this by introducing a causal version of Dilated Neighborhood Attention while incorporating it with convolutions. Our model achieves comparable, better, or state-of-the-art performance over TCNs, TCAN, LSTMs, and GRUs while requiring fewer parameters on standard emotion recognition datasets.
  arXiv  Detail & Related papers  (2023-12-12T18:41:30Z)
• Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos [63.94040814459116]
  Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. We propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations.
  arXiv  Detail & Related papers  (2023-08-19T09:12:13Z)
• Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network [52.29330138835208]
  Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images. We propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide message passing.
  arXiv  Detail & Related papers  (2023-07-04T02:50:44Z)
• LOGO-Former: Local-Global Spatio-Temporal Transformer for Dynamic Facial Expression Recognition [19.5702895176141]
  Previous methods for facial expression recognition (DFER) in the wild are mainly based on Convolutional Neural Networks (CNNs), whose local operations ignore the long-range dependencies in videos. We propose Transformer-based methods for DFER to achieve better performances but result in higher FLOPs and computational costs. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and FERV39K) indicate that our method provides an effective way to make use of the spatial and temporal dependencies for DFER.
  arXiv  Detail & Related papers  (2023-05-05T07:53:13Z)
• Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos [78.45050529204701]
  We propose a novel framework to pursue discriminative and robust representation by modeling cross-scale spatial-temporal correlation. CTL utilizes a CNN backbone and a key-points estimator to extract semantic local features from human body. It explores a context-reinforced topology to construct multi-scale graphs by considering both global contextual information and physical connections of human body.
  arXiv  Detail & Related papers  (2021-04-15T14:32:12Z)
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations [72.4716073597902]
  We propose a method to learn object Canonical Point Cloud Representations of dynamically or moving objects. We demonstrate the effectiveness of our method on several applications including shape reconstruction, camera pose estimation, continuoustemporal sequence reconstruction, and correspondence estimation.
  arXiv  Detail & Related papers  (2020-08-06T17:58:48Z)
• Co-Saliency Spatio-Temporal Interaction Network for Person Re-Identification in Videos [85.6430597108455]
  We propose a novel Co-Saliency Spatio-Temporal Interaction Network (CSTNet) for person re-identification in videos. It captures the common salient foreground regions among video frames and explores the spatial-temporal long-range context interdependency from such regions. Multiple spatialtemporal interaction modules within CSTNet are proposed, which exploit the spatial and temporal long-range context interdependencies on such features and spatial-temporal information correlation.
  arXiv  Detail & Related papers  (2020-04-10T10:23:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.