Temporal-Channel Topology Enhanced Network for Skeleton-Based Action Recognition

• URL: http://arxiv.org/abs/2302.12967v1
• Date: Sat, 25 Feb 2023 03:09:07 GMT
• Title: Temporal-Channel Topology Enhanced Network for Skeleton-Based Action Recognition
• Authors: Jinzhao Luo, Lu Zhou, Guibo Zhu, Guojing Ge, Beiying Yang, Jinqiao Wang
• Abstract summary: We propose a novel CNN architecture, Temporal-Channel Topology Enhanced Network (TCTE-Net), to learn spatial and temporal topologies for skeleton-based action recognition. TCTE-Net shows state-of-the-art performance compared to CNN-based methods and achieves superior performance compared to GCN-based methods.
• Score: 26.609509266693077
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Skeleton-based action recognition has become popular in recent years due to its efficiency and robustness. Most current methods adopt graph convolutional network (GCN) for topology modeling, but GCN-based methods are limited in long-distance correlation modeling and generalizability. In contrast, the potential of convolutional neural network (CNN) for topology modeling has not been fully explored. In this paper, we propose a novel CNN architecture, Temporal-Channel Topology Enhanced Network (TCTE-Net), to learn spatial and temporal topologies for skeleton-based action recognition. The TCTE-Net consists of two modules: the Temporal-Channel Focus module, which learns a temporal-channel focus matrix to identify the most critical feature representations, and the Dynamic Channel Topology Attention module, which dynamically learns spatial topological features, and fuses them with an attention mechanism to model long-distance channel-wise topology. We conduct experiments on NTU RGB+D, NTU RGB+D 120, and FineGym datasets. TCTE-Net shows state-of-the-art performance compared to CNN-based methods and achieves superior performance compared to GCN-based methods. The code is available at https://github.com/aikuniverse/TCTE-Net.

Related papers

• TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals [58.865901821451295]
  We present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form.
  arXiv  Detail & Related papers  (2024-04-15T06:01:48Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Interference Cancellation GAN Framework for Dynamic Channels [74.22393885274728]
  We introduce an online training framework that can adapt to any changes in the channel. Our framework significantly outperforms recent neural network models on highly dynamic channels.
  arXiv  Detail & Related papers  (2022-08-17T02:01:18Z)
• Skeleton-based Action Recognition via Temporal-Channel Aggregation [5.620303498964992]
  We propose a Temporal-Channel Aggregation Graph Conal Networks (TCA-CN) to learn spatial and temporal topologies. In addition, we extract multi-scale skeletal temporal modeling and fuse them with priori skeletal knowledge with an attention mechanism.
  arXiv  Detail & Related papers  (2022-05-31T16:28:30Z)
• Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition [15.93566875893684]
  We propose a pure CNN architecture named Topology-aware CNN (Ta-CNN) in this paper. We develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations. In particular, we develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations.
  arXiv  Detail & Related papers  (2021-12-08T09:02:50Z)
• Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition [40.103229224732196]
  We propose a novel Channel-wise Topology Refinement Graph Convolution (CTR-GC) to learn different topologies. Our refinement method introduces few extra parameters and significantly reduces the difficulty of modeling channel-wise topologies. We develop a powerful graph convolutional network named CTR-GCN which notably outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-07-26T13:37:50Z)
• Overcoming Catastrophic Forgetting in Graph Neural Networks [50.900153089330175]
  Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. We propose a novel scheme dedicated to overcoming this problem and hence strengthen continual learning in graph neural networks (GNNs) At the heart of our approach is a generic module, termed as topology-aware weight preserving(TWP)
  arXiv  Detail & Related papers  (2020-12-10T22:30:25Z)
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition [126.51241919472356]
  We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition. Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
  arXiv  Detail & Related papers  (2020-11-26T14:43:04Z)
• On the spatial attention in Spatio-Temporal Graph Convolutional Networks for skeleton-based human action recognition [97.14064057840089]
  Graphal networks (GCNs) promising performance in skeleton-based human action recognition by modeling a sequence of skeletons as a graph. Most of the recently proposed G-temporal-based methods improve the performance by learning the graph structure at each layer of the network.
  arXiv  Detail & Related papers  (2020-11-07T19:03:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.