On the spatial attention in Spatio-Temporal Graph Convolutional Networks for skeleton-based human action recognition

• URL: http://arxiv.org/abs/2011.03833v2
• Date: Thu, 22 Apr 2021 17:53:15 GMT
• Title: On the spatial attention in Spatio-Temporal Graph Convolutional Networks for skeleton-based human action recognition
• Authors: Negar Heidari, Alexandros Iosifidis
• Abstract summary: Graphal networks (GCNs) promising performance in skeleton-based human action recognition by modeling a sequence of skeletons as a graph. Most of the recently proposed G-temporal-based methods improve the performance by learning the graph structure at each layer of the network.
• Score: 97.14064057840089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph convolutional networks (GCNs) achieved promising performance in skeleton-based human action recognition by modeling a sequence of skeletons as a spatio-temporal graph. Most of the recently proposed GCN-based methods improve the performance by learning the graph structure at each layer of the network using a spatial attention applied on a predefined graph Adjacency matrix that is optimized jointly with model's parameters in an end-to-end manner. In this paper, we analyze the spatial attention used in spatio-temporal GCN layers and propose a symmetric spatial attention for better reflecting the symmetric property of the relative positions of the human body joints when executing actions. We also highlight the connection of spatio-temporal GCN layers employing additive spatial attention to bilinear layers, and we propose the spatio-temporal bilinear network (ST-BLN) which does not require the use of predefined Adjacency matrices and allows for more flexible design of the model. Experimental results show that the three models lead to effectively the same performance. Moreover, by exploiting the flexibility provided by the proposed ST-BLN, one can increase the efficiency of the model.

Related papers

• Multi-Scale Spatial-Temporal Self-Attention Graph Convolutional Networks for Skeleton-based Action Recognition [0.0]
  In this paper, we propose self-attention GCN hybrid model, Multi-Scale Spatial-Temporal self-attention (MSST)-GCN. We utilize spatial self-attention module with adaptive topology to understand intra-frame interactions within a frame among different body parts, and temporal self-attention module to examine correlations between frames of a node.
  arXiv  Detail & Related papers  (2024-04-03T10:25:45Z)
• DD-GCN: Directed Diffusion Graph Convolutional Network for Skeleton-based Human Action Recognition [10.115283931959855]
  Graphal Networks (GCNs) have been widely used in skeleton-based human action recognition. In this paper, we construct directed diffusion for action modeling and introduce the activity partition strategy. We also present to embed synchronized-temporal-temporal semantics.
  arXiv  Detail & Related papers  (2023-08-24T01:53:59Z)
• DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition [77.87404524458809]
  We propose a new framework for skeleton-based action recognition, namely Dynamic Group Spatio-Temporal GCN (DG-STGCN) It consists of two modules, DG-GCN and DG-TCN, respectively, for spatial and temporal modeling. DG-STGCN consistently outperforms state-of-the-art methods, often by a notable margin.
  arXiv  Detail & Related papers  (2022-10-12T03:17:37Z)
• Spatio-Temporal Joint Graph Convolutional Networks for Traffic Forecasting [75.10017445699532]
  Recent have shifted their focus towards formulating traffic forecasting as atemporal graph modeling problem. We propose a novel approach for accurate traffic forecasting on road networks over multiple future time steps.
  arXiv  Detail & Related papers  (2021-11-25T08:45:14Z)
• Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning [9.88508686848173]
  Biological spiking neurons with intrinsic dynamics underlie the powerful representation and learning capabilities of the brain. Despite recent tremendous progress in spiking neural networks (SNNs) for handling Euclidean-space tasks, it still remains challenging to exploit SNNs in processing non-Euclidean-space data. Here we present a general spike-based modeling framework that enables the direct training of SNNs for graph learning.
  arXiv  Detail & Related papers  (2021-06-30T11:20:16Z)
• A Comparative Study of Using Spatial-Temporal Graph Convolutional Networks for Predicting Availability in Bike Sharing Schemes [13.819341724635319]
  We present an Attention-based ST-GCN (AST-GCN) for predicting the number of available bikes in bike-sharing systems in cities. Our experimental results are presented using two real-world datasets, Dublinbikes and NYC-Citi Bike.
  arXiv  Detail & Related papers  (2021-04-21T17:13:29Z)
• Mix Dimension in Poincar\'{e} Geometry for 3D Skeleton-based Action Recognition [57.98278794950759]
  Graph Convolutional Networks (GCNs) have already demonstrated their powerful ability to model the irregular data. We present a novel spatial-temporal GCN architecture which is defined via the Poincar'e geometry. We evaluate our method on two current largest scale 3D datasets.
  arXiv  Detail & Related papers  (2020-07-30T18:23:18Z)
• Binarized Graph Neural Network [65.20589262811677]
  We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters. Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches. Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
  arXiv  Detail & Related papers  (2020-04-19T09:43:14Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)
• Feedback Graph Convolutional Network for Skeleton-based Action Recognition [38.782491442635205]
  We propose a novel network, named Feedback Graph Convolutional Network (FGCN) This is the first work that introduces the feedback mechanism into GCNs and action recognition. It achieves the state-of-the-art performance on three datasets.
  arXiv  Detail & Related papers  (2020-03-17T07:20:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.