Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes

• URL: http://arxiv.org/abs/2110.13408v1
• Date: Tue, 26 Oct 2021 04:42:24 GMT
• Title: Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes
• Authors: Yunjie Peng, Saihui Hou, Kang Ma, Yang Zhang, Yongzhen Huang, Zhiqiang He
• Abstract summary: This paper proposes a simple yet effective bimodal fusion network, which mines the complementary clues of skeletons and silhouettes to learn rich features for gait identification. Under the most challenging condition of walking in different clothes on CASIA-B, our method achieves the rank-1 accuracy of 92.1%.
• Score: 20.766540020533803
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gait recognition captures gait patterns from the walking sequence of an individual for identification. Most existing gait recognition methods learn features from silhouettes or skeletons for the robustness to clothing, carrying, and other exterior factors. The combination of the two data modalities, however, is not fully exploited. This paper proposes a simple yet effective bimodal fusion (BiFusion) network, which mines the complementary clues of skeletons and silhouettes, to learn rich features for gait identification. Particularly, the inherent hierarchical semantics of body joints in a skeleton is leveraged to design a novel Multi-scale Gait Graph (MSGG) network for the feature extraction of skeletons. Extensive experiments on CASIA-B and OUMVLP demonstrate both the superiority of the proposed MSGG network in modeling skeletons and the effectiveness of the bimodal fusion for gait recognition. Under the most challenging condition of walking in different clothes on CASIA-B, our method achieves the rank-1 accuracy of 92.1%.

Related papers

• ZipGait: Bridging Skeleton and Silhouette with Diffusion Model for Advancing Gait Recognition [31.732554267037305]
  We make the first attempt to reconstruct dense body shapes from discrete skeleton distributions via the diffusion model. We introduce Perception Gait Integration (PGI) to integrate different gait features through a two-stage process. ZipGait demonstrates superior performance, outperforming the state-of-the-art methods by a large margin under both cross-domain and intra-domain settings.
  arXiv  Detail & Related papers  (2024-08-22T03:52:44Z)
• GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition [26.721242606715354]
  Gait recognition is a biometric technology that recognizes the identity of humans through their walking patterns. We propose a novel gait recognition framework, dubbed Gait Multi-model Aggregation Network (GaitMA) First, skeletons are represented by joint/limb-based heatmaps, and features from silhouettes and skeletons are respectively extracted using two CNN-based feature extractors.
  arXiv  Detail & Related papers  (2024-07-20T09:05:17Z)
• TriGait: Aligning and Fusing Skeleton and Silhouette Gait Data via a Tri-Branch Network [4.699718818019937]
  Gait recognition is a promising biometric technology for identification due to its non-invasiveness and long-distance. external variations such as clothing changes and viewpoint differences pose significant challenges to gait recognition. A novel triple branch gait recognition framework, TriGait, is proposed in this paper.
  arXiv  Detail & Related papers  (2023-08-25T12:19:51Z)
• Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard Skeleton Mining for Unsupervised Person Re-Identification [70.90142717649785]
  This paper proposes a generic unsupervised Hierarchical skeleton Meta-Prototype Contrastive learning (Hi-MPC) approach with Hard Skeleton Mining (HSM) for person re-ID with unlabeled 3D skeletons. By converting original prototypes into meta-prototypes with multiple homogeneous transformations, we induce the model to learn the inherent consistency of prototypes to capture more effective skeleton features for person re-ID.
  arXiv  Detail & Related papers  (2023-07-24T16:18:22Z)
• SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training [110.55093254677638]
  We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL) In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE. Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
  arXiv  Detail & Related papers  (2023-07-17T13:33:11Z)
• One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching [77.6989219290789]
  One-shot skeleton action recognition aims to learn a skeleton action recognition model with a single training sample. This paper presents a novel one-shot skeleton action recognition technique that handles skeleton action recognition via multi-scale spatial-temporal feature matching.
  arXiv  Detail & Related papers  (2023-07-14T11:52:10Z)
• Multi-Modal Human Authentication Using Silhouettes, Gait and RGB [59.46083527510924]
  Whole-body-based human authentication is a promising approach for remote biometrics scenarios. We propose Dual-Modal Ensemble (DME), which combines both RGB and silhouette data to achieve more robust performances for indoor and outdoor whole-body based recognition. Within DME, we propose GaitPattern, which is inspired by the double helical gait pattern used in traditional gait analysis.
  arXiv  Detail & Related papers  (2022-10-08T15:17:32Z)
• Skeleton-based Action Recognition via Adaptive Cross-Form Learning [75.92422282666767]
  Skeleton-based action recognition aims to project skeleton sequences to action categories, where sequences are derived from multiple forms of pre-detected points. Existing methods tend to improve GCNs by leveraging multi-form skeletons due to their complementary cues. We present Adaptive Cross-Form Learning (ACFL), which empowers well-designed GCNs to generate complementary representation from single-form skeletons.
  arXiv  Detail & Related papers  (2022-06-30T07:40:03Z)
• SimMC: Simple Masked Contrastive Learning of Skeleton Representations for Unsupervised Person Re-Identification [63.903237777588316]
  We present a generic Simple Masked Contrastive learning (SimMC) framework to learn effective representations from unlabeled 3D skeletons for person re-ID. Specifically, to fully exploit skeleton features within each skeleton sequence, we first devise a masked prototype contrastive learning (MPC) scheme. Then, we propose the masked intra-sequence contrastive learning (MIC) to capture intra-sequence pattern consistency between subsequences.
  arXiv  Detail & Related papers  (2022-04-21T00:19:38Z)
• Towards a Deeper Understanding of Skeleton-based Gait Recognition [4.812321790984493]
  In recent years, most gait recognition methods used the person's silhouette to extract the gait features. Model-based methods do not suffer from these problems and are able to represent the temporal motion of body joints. In this work, we propose an approach based on Graph Convolutional Networks (GCNs) that combines higher-order inputs, and residual networks.
  arXiv  Detail & Related papers  (2022-04-16T18:23:37Z)
• Combining the Silhouette and Skeleton Data for Gait Recognition [13.345465199699]
  Two dominant gait recognition works are appearance-based and model-based, which extract features from silhouettes and skeletons, respectively. This paper proposes a CNN-based branch taking silhouettes as input and a GCN-based branch taking skeletons as input. For better gait representation in the GCN-based branch, we present a fully connected graph convolution operator to integrate multi-scale graph convolutions.
  arXiv  Detail & Related papers  (2022-02-22T03:21:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.