Improving Skeleton-based Action Recognitionwith Robust Spatial and Temporal Features

• URL: http://arxiv.org/abs/2008.00324v1
• Date: Sat, 1 Aug 2020 19:29:53 GMT
• Title: Improving Skeleton-based Action Recognitionwith Robust Spatial and Temporal Features
• Authors: Zeshi Yang and Kangkang Yin
• Abstract summary: We propose a novel mechanism to learn more robust discriminative features in space and time. We show thataction recognition accuracy can be improved when these robust featuresare learned and used.
• Score: 6.548580592686076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently skeleton-based action recognition has made signif-icant progresses in the computer vision community. Most state-of-the-art algorithms are based on Graph Convolutional Networks (GCN), andtarget at improving the network structure of the backbone GCN lay-ers. In this paper, we propose a novel mechanism to learn more robustdiscriminative features in space and time. More specifically, we add aDiscriminative Feature Learning (DFL) branch to the last layers of thenetwork to extract discriminative spatial and temporal features to helpregularize the learning. We also formally advocate the use of Direction-Invariant Features (DIF) as input to the neural networks. We show thataction recognition accuracy can be improved when these robust featuresare learned and used. We compare our results with those of ST-GCNand related methods on four datasets: NTU-RGBD60, NTU-RGBD120,SYSU 3DHOI and Skeleton-Kinetics.

Related papers

• Signal-SGN: A Spiking Graph Convolutional Network for Skeletal Action Recognition via Learning Temporal-Frequency Dynamics [2.9578022754506605]
  In skeletal-based action recognition, Graph Convolutional Networks (GCNs) face limitations due to their complexity and high energy consumption. We propose a Signal-SGN(Spiking Graph Convolutional Network), which leverages the temporal dimension of skeletal sequences as the spiking timestep. Our experiments show that the proposed models not only surpass existing SNN-based methods in accuracy but also reduce computational storage costs during training.
  arXiv  Detail & Related papers  (2024-08-03T07:47:16Z)
• Enhancing Graph Representation Learning with Attention-Driven Spiking Neural Networks [5.627287101959473]
  Spiking neural networks (SNNs) have emerged as a promising alternative to traditional neural networks for graph learning tasks. We propose a novel approach that integrates attention mechanisms with SNNs to improve graph representation learning.
  arXiv  Detail & Related papers  (2024-03-25T12:15:10Z)
• Unsupervised Spatial-Temporal Feature Enrichment and Fidelity Preservation Network for Skeleton based Action Recognition [20.07820929037547]
  Unsupervised skeleton based action recognition has achieved remarkable progress recently. Existing unsupervised learning methods suffer from severe overfitting problem. This paper presents an Unsupervised spatial-temporal Feature Enrichment and Fidelity Preservation framework to generate rich distributed features.
  arXiv  Detail & Related papers  (2024-01-25T09:24:07Z)
• Graph Neural Networks Provably Benefit from Structural Information: A Feature Learning Perspective [53.999128831324576]
  Graph neural networks (GNNs) have pioneered advancements in graph representation learning. This study investigates the role of graph convolution within the context of feature learning theory.
  arXiv  Detail & Related papers  (2023-06-24T10:21:11Z)
• Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognition [71.35205097460124]
  How humans understand and recognize the actions of others is a complex neuroscientific problem. LA-GCN proposes a graph convolution network using large-scale language models (LLM) knowledge assistance.
  arXiv  Detail & Related papers  (2023-05-21T08:29:16Z)
• Spatial Temporal Graph Convolution with Graph Structure Self-learning for Early MCI Detection [9.11430195887347]
  We propose a spatial temporal graph convolutional network with a novel graph structure self-learning mechanism for EMCI detection. Results on the Alzheimer's Disease Neuroimaging Initiative database show that our method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-11T12:29:00Z)
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition [126.51241919472356]
  We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition. Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
  arXiv  Detail & Related papers  (2020-11-26T14:43:04Z)
• A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification [65.18004601366066]
  Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID.
  arXiv  Detail & Related papers  (2020-09-05T16:06:04Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)
• Feedback Graph Convolutional Network for Skeleton-based Action Recognition [38.782491442635205]
  We propose a novel network, named Feedback Graph Convolutional Network (FGCN) This is the first work that introduces the feedback mechanism into GCNs and action recognition. It achieves the state-of-the-art performance on three datasets.
  arXiv  Detail & Related papers  (2020-03-17T07:20:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.