GCsT: Graph Convolutional Skeleton Transformer for Action Recognition

• URL: http://arxiv.org/abs/2109.02860v2
• Date: Wed, 8 Sep 2021 04:03:35 GMT
• Title: GCsT: Graph Convolutional Skeleton Transformer for Action Recognition
• Abstract summary: Graph convolutional networks (GCNs) achieve promising performance for skeleton-based action recognition. In most GCN-based methods, the spatial-temporal graph convolution is strictly restricted by the graph topology. We present a novel architecture, named Graph Convolutional skeleton Transformer (GCsT), which addresses limitations in GCNs by introducing Transformer.
• Score: 10.49514906434189
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph convolutional networks (GCNs) achieve promising performance for skeleton-based action recognition. However, in most GCN-based methods, the spatial-temporal graph convolution is strictly restricted by the graph topology while only captures the short-term temporal context, thus lacking the flexibility of feature extraction. In this work, we present a novel architecture, named Graph Convolutional skeleton Transformer (GCsT), which addresses limitations in GCNs by introducing Transformer. Our GCsT employs all the benefits of Transformer (i.e. dynamical attention and global context) while keeps the advantages of GCNs (i.e. hierarchy and local topology structure). In GCsT, the spatial-temporal GCN forces the capture of local dependencies while Transformer dynamically extracts global spatial-temporal relationships. Furthermore, the proposed GCsT shows stronger expressive capability by adding additional information present in skeleton sequences. Incorporating the Transformer allows that information to be introduced into the model almost effortlessly. We validate the proposed GCsT by conducting extensive experiments, which achieves the state-of-the-art performance on NTU RGB+D, NTU RGB+D 120 and Northwestern-UCLA datasets.

Related papers

• Size Transferability of Graph Transformers with Convolutional Positional Encodings [82.27361992510494]
  Graph Transformers (GTs) are attention-based architectures for graph-structured data.<n>We study GTs through the lens of manifold limit models for graph sequences.<n>We show that GTs inherit transferability guarantees from their positional encodings.
  arXiv  Detail & Related papers  (2026-02-16T22:38:56Z)
• CGTGait: Collaborative Graph and Transformer for Gait Emotion Recognition [9.539530757201206]
  Skeleton-based gait emotion recognition has received significant attention due to its wide-ranging applications.<n>In this paper, we propose textbfCGTGait a framework that collaboratively integrates graph convolution and transformers.<n>We evaluate our method on two widely used datasets, Emotion-Gait and ELMD.
  arXiv  Detail & Related papers  (2025-09-20T10:48:51Z)
• Towards Pre-trained Graph Condensation via Optimal Transport [52.6504753271008]
  Graph condensation aims to distill the original graph into a small-scale graph, mitigating redundancy and accelerating GNN training.<n> conventional GC approaches heavily rely on rigid GNNs and task-specific supervision.<n>Pre-trained Graph Condensation (PreGC) via optimal transport is proposed to transcend the limitations of task- and architecture-dependent GC methods.
  arXiv  Detail & Related papers  (2025-09-18T08:13:24Z)
• Position-aware Graph Transformer for Recommendation [13.448664870389722]
  We propose a new graph transformer (GT) framework -- textitPosition-aware Graph Transformer for Recommendation (PGTR) The key insight is to explicitly incorporate node position and structure information from the user-item interaction graph into GT architecture. Empirical studies demonstrate the effectiveness of the proposed PGTR method when implemented on various GCN-based backbones across four real-world datasets.
  arXiv  Detail & Related papers  (2024-12-25T01:22:35Z)
• Point Cloud Denoising With Fine-Granularity Dynamic Graph Convolutional Networks [58.050130177241186]
  Noise perturbations often corrupt 3-D point clouds, hindering downstream tasks such as surface reconstruction, rendering, and further processing. This paper introduces finegranularity dynamic graph convolutional networks called GDGCN, a novel approach to denoising in 3-D point clouds.
  arXiv  Detail & Related papers  (2024-11-21T14:19:32Z)
• Diss-l-ECT: Dissecting Graph Data with local Euler Characteristic Transforms [13.608942872770855]
  We introduce the Local Euler Characteristic Transform ($ell$-ECT) to enhance expressivity and interpretability in graph representation learning. Unlike traditional Graph Neural Networks (GNNs), which may lose critical local details through aggregation, the $ell$-ECT provides a lossless representation of local neighborhoods. Our method exhibits superior performance than standard GNNs on a variety of node classification tasks, particularly in graphs with high heterophily.
  arXiv  Detail & Related papers  (2024-10-03T16:02:02Z)
• Signal-SGN: A Spiking Graph Convolutional Network for Skeletal Action Recognition via Learning Temporal-Frequency Dynamics [2.9578022754506605]
  In skeletal-based action recognition, Graph Convolutional Networks (GCNs) face limitations due to their complexity and high energy consumption. We propose a Signal-SGN(Spiking Graph Convolutional Network), which leverages the temporal dimension of skeletal sequences as the spiking timestep. Our experiments show that the proposed models not only surpass existing SNN-based methods in accuracy but also reduce computational storage costs during training.
  arXiv  Detail & Related papers  (2024-08-03T07:47:16Z)
• Scalable Graph Compressed Convolutions [68.85227170390864]
  We propose a differentiable method that applies permutations to calibrate input graphs for Euclidean convolution. Based on the graph calibration, we propose the Compressed Convolution Network (CoCN) for hierarchical graph representation learning.
  arXiv  Detail & Related papers  (2024-07-26T03:14:13Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• ASWT-SGNN: Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network [20.924559944655392]
  This paper proposes an Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network (ASWT-SGNN) ASWT-SGNN accurately approximates the filter function in high-density spectral regions, avoiding costly eigen-decomposition. It achieves comparable performance to state-of-the-art models in node classification tasks.
  arXiv  Detail & Related papers  (2023-12-10T03:07:42Z)
• STEP CATFormer: Spatial-Temporal Effective Body-Part Cross Attention Transformer for Skeleton-based Action Recognition [0.0]
  We focus on how the Graph Convolutional Convolution networks learn different topologies and effectively aggregate joint features in the global temporal and local temporal. We propose three Channel-wise Tolopogy Graph Convolution based on Channel-wise Topology Refinement Graph Convolution (CTR-GCN) We develop a powerful graph convolutional network named Spatial Temporal Effective Body-part Cross Attention Transformer which notably high-performance on the NTU RGB+D, NTU RGB+D 120 datasets.
  arXiv  Detail & Related papers  (2023-12-06T04:36:58Z)
• Adaptive Graph Convolution Networks for Traffic Flow Forecasting [4.398745005061698]
  We propose a novel Adaptive Graph Convolution Networks (AGC-net) to address this issue in Graph neural networks (GNNs) The AGC-net is constructed by the Adaptive Graph Convolution (AGC) based on a novel context attention mechanism. Experimental results on two public traffic datasets demonstrate the effectiveness of the AGC-net.
  arXiv  Detail & Related papers  (2023-07-07T09:55:41Z)
• Is Attention All NeRF Needs? [103.51023982774599]
  Generalizable NeRF Transformer (GNT) is a pure, unified transformer-based architecture that efficiently reconstructs Neural Radiance Fields (NeRFs) on the fly from source views. GNT achieves generalizable neural scene representation and rendering, by encapsulating two transformer-based stages.
  arXiv  Detail & Related papers  (2022-07-27T05:09:54Z)
• Orthogonal Graph Neural Networks [53.466187667936026]
  Graph neural networks (GNNs) have received tremendous attention due to their superiority in learning node representations. stacking more convolutional layers significantly decreases the performance of GNNs. We propose a novel Ortho-GConv, which could generally augment the existing GNN backbones to stabilize the model training and improve the model's generalization performance.
  arXiv  Detail & Related papers  (2021-09-23T12:39:01Z)
• Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition [40.103229224732196]
  We propose a novel Channel-wise Topology Refinement Graph Convolution (CTR-GC) to learn different topologies. Our refinement method introduces few extra parameters and significantly reduces the difficulty of modeling channel-wise topologies. We develop a powerful graph convolutional network named CTR-GCN which notably outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-07-26T13:37:50Z)
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition [126.51241919472356]
  We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition. Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
  arXiv  Detail & Related papers  (2020-11-26T14:43:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.