SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework
for Person Re-Identification
- URL: http://arxiv.org/abs/2107.01903v1
- Date: Mon, 5 Jul 2021 09:53:08 GMT
- Title: SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework
for Person Re-Identification
- Authors: Haocong Rao, Xiping Hu, Jun Cheng, Bin Hu
- Abstract summary: Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications.
Existing methods typically learn body and motion features from the body-joint trajectory.
We propose a Self-supervised Multi-scale Graph (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics.
- Score: 12.347724797202865
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Person re-identification via 3D skeletons is an emerging topic with great
potential in security-critical applications. Existing methods typically learn
body and motion features from the body-joint trajectory, whereas they lack a
systematic way to model body structure and underlying relations of body
components beyond the scale of body joints. In this paper, we for the first
time propose a Self-supervised Multi-scale Skeleton Graph Encoding (SM-SGE)
framework that comprehensively models human body, component relations, and
skeleton dynamics from unlabeled skeleton graphs of various scales to learn an
effective skeleton representation for person Re-ID. Specifically, we first
devise multi-scale skeleton graphs with coarse-to-fine human body partitions,
which enables us to model body structure and skeleton dynamics at multiple
levels. Second, to mine inherent correlations between body components in
skeletal motion, we propose a multi-scale graph relation network to learn
structural relations between adjacent body-component nodes and collaborative
relations among nodes of different scales, so as to capture more discriminative
skeleton graph features. Last, we propose a novel multi-scale skeleton
reconstruction mechanism to enable our framework to encode skeleton dynamics
and high-level semantics from unlabeled skeleton graphs, which encourages
learning a discriminative skeleton representation for person Re-ID. Extensive
experiments show that SM-SGE outperforms most state-of-the-art skeleton-based
methods. We further demonstrate its effectiveness on 3D skeleton data estimated
from large-scale RGB videos. Our codes are open at
https://github.com/Kali-Hac/SM-SGE.
Related papers
- SkeleTR: Towrads Skeleton-based Action Recognition in the Wild [86.03082891242698]
SkeleTR is a new framework for skeleton-based action recognition.
It first models the intra-person skeleton dynamics for each skeleton sequence with graph convolutions.
It then uses stacked Transformer encoders to capture person interactions that are important for action recognition in general scenarios.
arXiv Detail & Related papers (2023-09-20T16:22:33Z) - Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard
Skeleton Mining for Unsupervised Person Re-Identification [70.90142717649785]
This paper proposes a generic unsupervised Hierarchical skeleton Meta-Prototype Contrastive learning (Hi-MPC) approach with Hard Skeleton Mining (HSM) for person re-ID with unlabeled 3D skeletons.
By converting original prototypes into meta-prototypes with multiple homogeneous transformations, we induce the model to learn the inherent consistency of prototypes to capture more effective skeleton features for person re-ID.
arXiv Detail & Related papers (2023-07-24T16:18:22Z) - SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence
Pre-training [110.55093254677638]
We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL)
In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE.
Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
arXiv Detail & Related papers (2023-07-17T13:33:11Z) - TranSG: Transformer-Based Skeleton Graph Prototype Contrastive Learning
with Structure-Trajectory Prompted Reconstruction for Person
Re-Identification [63.903237777588316]
Person re-identification (re-ID) via 3D skeleton data is an emerging topic with prominent advantages.
Existing methods usually design skeleton descriptors with raw body joints or perform skeleton sequence representation learning.
We propose a generic Transformer-based Skeleton Graph prototype contrastive learning (TranSG) approach with structure-trajectory prompted reconstruction.
arXiv Detail & Related papers (2023-03-13T02:27:45Z) - Skeleton Prototype Contrastive Learning with Multi-Level Graph Relation
Modeling for Unsupervised Person Re-Identification [63.903237777588316]
Person re-identification (re-ID) via 3D skeletons is an important emerging topic with many merits.
Existing solutions rarely explore valuable body-component relations in skeletal structure or motion.
This paper proposes a generic unsupervised Prototype Contrastive learning paradigm with Multi-level Graph Relation learning.
arXiv Detail & Related papers (2022-08-25T00:59:32Z) - SimMC: Simple Masked Contrastive Learning of Skeleton Representations
for Unsupervised Person Re-Identification [63.903237777588316]
We present a generic Simple Masked Contrastive learning (SimMC) framework to learn effective representations from unlabeled 3D skeletons for person re-ID.
Specifically, to fully exploit skeleton features within each skeleton sequence, we first devise a masked prototype contrastive learning (MPC) scheme.
Then, we propose the masked intra-sequence contrastive learning (MIC) to capture intra-sequence pattern consistency between subsequences.
arXiv Detail & Related papers (2022-04-21T00:19:38Z) - Skeleton-Contrastive 3D Action Representation Learning [35.06361753065124]
This paper strives for self-supervised learning of a feature space suitable for skeleton-based action recognition.
Our approach achieves state-of-the-art performance for self-supervised learning from skeleton data on the challenging PKU and NTU datasets.
arXiv Detail & Related papers (2021-08-08T14:44:59Z) - Multi-Level Graph Encoding with Structural-Collaborative Relation
Learning for Skeleton-Based Person Re-Identification [11.303008512400893]
Skeleton-based person re-identification (Re-ID) is an emerging open topic providing great value for safety-critical applications.
Existing methods typically extract hand-crafted features or model skeleton dynamics from the trajectory of body joints.
We propose a Multi-level Graph encoding approach with Structural-Collaborative Relation learning (MG-SCR) to encode discriminative graph features for person Re-ID.
arXiv Detail & Related papers (2021-06-06T09:09:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.