Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition

• URL: http://arxiv.org/abs/2112.04178v2
• Date: Thu, 9 Dec 2021 02:42:44 GMT
• Title: Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition
• Authors: Kailin Xu, Fanfan Ye, Qiaoyong Zhong, Di Xie
• Abstract summary: We propose a pure CNN architecture named Topology-aware CNN (Ta-CNN) in this paper. We develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations. In particular, we develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations.
• Score: 15.93566875893684
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the context of skeleton-based action recognition, graph convolutional networks (GCNs) have been rapidly developed, whereas convolutional neural networks (CNNs) have received less attention. One reason is that CNNs are considered poor in modeling the irregular skeleton topology. To alleviate this limitation, we propose a pure CNN architecture named Topology-aware CNN (Ta-CNN) in this paper. In particular, we develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations. By applying the module to the coordinate level and the joint level subsequently, the topology feature is effectively enhanced. Notably, we theoretically prove that graph convolution is a special case of normal convolution when the joint dimension is treated as channels. This confirms that the topology modeling power of GCNs can also be implemented by using a CNN. Moreover, we creatively design a SkeletonMix strategy which mixes two persons in a unique manner and further boosts the performance. Extensive experiments are conducted on four widely used datasets, i.e. N-UCLA, SBU, NTU RGB+D and NTU RGB+D 120 to verify the effectiveness of Ta-CNN. We surpass existing CNN-based methods significantly. Compared with leading GCN-based methods, we achieve comparable performance with much less complexity in terms of the required GFLOPs and parameters.

Related papers

• Model Parallel Training and Transfer Learning for Convolutional Neural Networks by Domain Decomposition [0.0]
  Deep convolutional neural networks (CNNs) have been shown to be very successful in a wide range of image processing applications. Due to their increasing number of model parameters and an increasing availability of large amounts of training data, parallelization strategies to efficiently train complex CNNs are necessary.
  arXiv  Detail & Related papers  (2024-08-26T17:35:01Z)
• CNN2GNN: How to Bridge CNN with GNN [59.42117676779735]
  We propose a novel CNN2GNN framework to unify CNN and GNN together via distillation. The performance of distilled boosted'' two-layer GNN on Mini-ImageNet is much higher than CNN containing dozens of layers such as ResNet152.
  arXiv  Detail & Related papers  (2024-04-23T08:19:08Z)
• Temporal-Channel Topology Enhanced Network for Skeleton-Based Action Recognition [26.609509266693077]
  We propose a novel CNN architecture, Temporal-Channel Topology Enhanced Network (TCTE-Net), to learn spatial and temporal topologies for skeleton-based action recognition. TCTE-Net shows state-of-the-art performance compared to CNN-based methods and achieves superior performance compared to GCN-based methods.
  arXiv  Detail & Related papers  (2023-02-25T03:09:07Z)
• What Can Be Learnt With Wide Convolutional Neural Networks? [69.55323565255631]
  We study infinitely-wide deep CNNs in the kernel regime. We prove that deep CNNs adapt to the spatial scale of the target function. We conclude by computing the generalisation error of a deep CNN trained on the output of another deep CNN.
  arXiv  Detail & Related papers  (2022-08-01T17:19:32Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition [40.103229224732196]
  We propose a novel Channel-wise Topology Refinement Graph Convolution (CTR-GC) to learn different topologies. Our refinement method introduces few extra parameters and significantly reduces the difficulty of modeling channel-wise topologies. We develop a powerful graph convolutional network named CTR-GCN which notably outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-07-26T13:37:50Z)
• Overcoming Catastrophic Forgetting in Graph Neural Networks [50.900153089330175]
  Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. We propose a novel scheme dedicated to overcoming this problem and hence strengthen continual learning in graph neural networks (GNNs) At the heart of our approach is a generic module, termed as topology-aware weight preserving(TWP)
  arXiv  Detail & Related papers  (2020-12-10T22:30:25Z)
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition [126.51241919472356]
  We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition. Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
  arXiv  Detail & Related papers  (2020-11-26T14:43:04Z)
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution [57.635467829558664]
  We introduce a structural regularization across convolutional kernels in a CNN. We show that CNNs now maintain performance with dramatic reduction in parameters and computations.
  arXiv  Detail & Related papers  (2020-09-04T20:41:47Z)
• Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition [40.467040910143616]
  We propose Dynamic GCN, in which a novel convolutional neural network named Contextencoding Network (CeN) is introduced to learn skeleton topology automatically. CeN is extremely lightweight yet effective, and can be embedded into a graph convolutional layer. Dynamic GCN achieves better performance with $2times$$4times$ fewer FLOPs than existing methods.
  arXiv  Detail & Related papers  (2020-07-29T09:12:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.