SkeletonGait: Gait Recognition Using Skeleton Maps

• URL: http://arxiv.org/abs/2311.13444v2
• Date: Mon, 18 Dec 2023 09:15:39 GMT
• Title: SkeletonGait: Gait Recognition Using Skeleton Maps
• Authors: Chao Fan, Jingzhe Ma, Dongyang Jin, Chuanfu Shen, Shiqi Yu
• Abstract summary: We introduce a novel skeletal gait representation named skeleton map, together with SkeletonGait, a skeleton-based method to exploit structural information from human skeleton maps. Skeleton map represents the coordinates of human joints as a heatmap with Gaussian approximation, exhibiting a silhouette-like image devoid of exact body structure. SkeletonGait++ outperforms existing state-of-the-art methods by a significant margin in various scenarios.
• Score: 7.335859292188816
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The choice of the representations is essential for deep gait recognition methods. The binary silhouettes and skeletal coordinates are two dominant representations in recent literature, achieving remarkable advances in many scenarios. However, inherent challenges remain, in which silhouettes are not always guaranteed in unconstrained scenes, and structural cues have not been fully utilized from skeletons. In this paper, we introduce a novel skeletal gait representation named skeleton map, together with SkeletonGait, a skeleton-based method to exploit structural information from human skeleton maps. Specifically, the skeleton map represents the coordinates of human joints as a heatmap with Gaussian approximation, exhibiting a silhouette-like image devoid of exact body structure. Beyond achieving state-of-the-art performances over five popular gait datasets, more importantly, SkeletonGait uncovers novel insights about how important structural features are in describing gait and when they play a role. Furthermore, we propose a multi-branch architecture, named SkeletonGait++, to make use of complementary features from both skeletons and silhouettes. Experiments indicate that SkeletonGait++ outperforms existing state-of-the-art methods by a significant margin in various scenarios. For instance, it achieves an impressive rank-1 accuracy of over 85% on the challenging GREW dataset. All the source code is available at https://github.com/ShiqiYu/OpenGait.

Related papers

• GaitSTR: Gait Recognition with Sequential Two-stream Refinement [12.256802601846749]
  Gait recognition aims to identify a person based on their walking sequences, serving as a useful biometric modality. In representing a person's walking sequence, silhouettes and skeletons are the two primary modalities used. We explore the use of a two-stream representation of skeletons for gait recognition, alongside silhouettes.
  arXiv  Detail & Related papers  (2024-04-02T22:39:35Z)
• SkeleTR: Towrads Skeleton-based Action Recognition in the Wild [86.03082891242698]
  SkeleTR is a new framework for skeleton-based action recognition. It first models the intra-person skeleton dynamics for each skeleton sequence with graph convolutions. It then uses stacked Transformer encoders to capture person interactions that are important for action recognition in general scenarios.
  arXiv  Detail & Related papers  (2023-09-20T16:22:33Z)
• Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard Skeleton Mining for Unsupervised Person Re-Identification [70.90142717649785]
  This paper proposes a generic unsupervised Hierarchical skeleton Meta-Prototype Contrastive learning (Hi-MPC) approach with Hard Skeleton Mining (HSM) for person re-ID with unlabeled 3D skeletons. By converting original prototypes into meta-prototypes with multiple homogeneous transformations, we induce the model to learn the inherent consistency of prototypes to capture more effective skeleton features for person re-ID.
  arXiv  Detail & Related papers  (2023-07-24T16:18:22Z)
• SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training [110.55093254677638]
  We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL) In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE. Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
  arXiv  Detail & Related papers  (2023-07-17T13:33:11Z)
• Skeleton Prototype Contrastive Learning with Multi-Level Graph Relation Modeling for Unsupervised Person Re-Identification [63.903237777588316]
  Person re-identification (re-ID) via 3D skeletons is an important emerging topic with many merits. Existing solutions rarely explore valuable body-component relations in skeletal structure or motion. This paper proposes a generic unsupervised Prototype Contrastive learning paradigm with Multi-level Graph Relation learning.
  arXiv  Detail & Related papers  (2022-08-25T00:59:32Z)
• OSSO: Obtaining Skeletal Shape from Outside [50.47978215230605]
  OSSO (Obtaining Skeletal Shape from Outside) is the first to learn the mapping from the 3D body surface to the internal skeleton from real data. To these, we fit a parametric 3D body shape model (STAR) to capture the body surface and a novel part-based 3D skeleton model to capture the bones. We then train a regressor from body shape parameters to skeleton shape parameters and refine the skeleton to satisfy constraints on physical plausibility.
  arXiv  Detail & Related papers  (2022-04-21T14:33:42Z)
• SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification [12.347724797202865]
  Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory. We propose a Self-supervised Multi-scale Graph (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics.
  arXiv  Detail & Related papers  (2021-07-05T09:53:08Z)
• Point2Skeleton: Learning Skeletal Representations from Point Clouds [36.62519847312199]
  We introduce Point2Skeleton, an unsupervised method to learn skeletal representations from point clouds. We first predict a set of skeletal points by learning a geometric transformation, and then analyze the connectivity of the skeletal points to form skeletal mesh structures.
  arXiv  Detail & Related papers  (2020-12-01T03:04:09Z)
• SkeletonNet: A Topology-Preserving Solution for Learning Mesh Reconstruction of Object Surfaces from RGB Images [85.66560542483286]
  This paper focuses on the challenging task of learning 3D object surface reconstructions from RGB images. We propose two models, the Skeleton-Based GraphConvolutional Neural Network (SkeGCNN) and the Skeleton-Regularized Deep Implicit Surface Network (SkeDISN) We conduct thorough experiments that verify the efficacy of our proposed SkeletonNet.
  arXiv  Detail & Related papers  (2020-08-13T07:59:25Z)
• Predictively Encoded Graph Convolutional Network for Noise-Robust Skeleton-based Action Recognition [6.729108277517129]
  We propose a skeleton-based action recognition method which is robust to noise information of given skeleton features. Our approach achieves outstanding performance when skeleton samples are noised compared with existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-17T03:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.