Multi-Level Graph Encoding with Structural-Collaborative Relation
Learning for Skeleton-Based Person Re-Identification
- URL: http://arxiv.org/abs/2106.03069v1
- Date: Sun, 6 Jun 2021 09:09:57 GMT
- Title: Multi-Level Graph Encoding with Structural-Collaborative Relation
Learning for Skeleton-Based Person Re-Identification
- Authors: Haocong Rao, Shihao Xu, Xiping Hu, Jun Cheng, Bin Hu
- Abstract summary: Skeleton-based person re-identification (Re-ID) is an emerging open topic providing great value for safety-critical applications.
Existing methods typically extract hand-crafted features or model skeleton dynamics from the trajectory of body joints.
We propose a Multi-level Graph encoding approach with Structural-Collaborative Relation learning (MG-SCR) to encode discriminative graph features for person Re-ID.
- Score: 11.303008512400893
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Skeleton-based person re-identification (Re-ID) is an emerging open topic
providing great value for safety-critical applications. Existing methods
typically extract hand-crafted features or model skeleton dynamics from the
trajectory of body joints, while they rarely explore valuable relation
information contained in body structure or motion. To fully explore body
relations, we construct graphs to model human skeletons from different levels,
and for the first time propose a Multi-level Graph encoding approach with
Structural-Collaborative Relation learning (MG-SCR) to encode discriminative
graph features for person Re-ID. Specifically, considering that
structurally-connected body components are highly correlated in a skeleton, we
first propose a multi-head structural relation layer to learn different
relations of neighbor body-component nodes in graphs, which helps aggregate key
correlative features for effective node representations. Second, inspired by
the fact that body-component collaboration in walking usually carries
recognizable patterns, we propose a cross-level collaborative relation layer to
infer collaboration between different level components, so as to capture more
discriminative skeleton graph features. Finally, to enhance graph dynamics
encoding, we propose a novel self-supervised sparse sequential prediction task
for model pre-training, which facilitates encoding high-level graph semantics
for person Re-ID. MG-SCR outperforms state-of-the-art skeleton-based methods,
and it achieves superior performance to many multi-modal methods that utilize
extra RGB or depth features. Our codes are available at
https://github.com/Kali-Hac/MG-SCR.
Related papers
- SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence
Pre-training [110.55093254677638]
We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL)
In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE.
Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
arXiv Detail & Related papers (2023-07-17T13:33:11Z) - TranSG: Transformer-Based Skeleton Graph Prototype Contrastive Learning
with Structure-Trajectory Prompted Reconstruction for Person
Re-Identification [63.903237777588316]
Person re-identification (re-ID) via 3D skeleton data is an emerging topic with prominent advantages.
Existing methods usually design skeleton descriptors with raw body joints or perform skeleton sequence representation learning.
We propose a generic Transformer-based Skeleton Graph prototype contrastive learning (TranSG) approach with structure-trajectory prompted reconstruction.
arXiv Detail & Related papers (2023-03-13T02:27:45Z) - Skeleton Prototype Contrastive Learning with Multi-Level Graph Relation
Modeling for Unsupervised Person Re-Identification [63.903237777588316]
Person re-identification (re-ID) via 3D skeletons is an important emerging topic with many merits.
Existing solutions rarely explore valuable body-component relations in skeletal structure or motion.
This paper proposes a generic unsupervised Prototype Contrastive learning paradigm with Multi-level Graph Relation learning.
arXiv Detail & Related papers (2022-08-25T00:59:32Z) - Joint-bone Fusion Graph Convolutional Network for Semi-supervised
Skeleton Action Recognition [65.78703941973183]
We propose a novel correlation-driven joint-bone fusion graph convolutional network (CD-JBF-GCN) as an encoder and use a pose prediction head as a decoder.
Specifically, the CD-JBF-GC can explore the motion transmission between the joint stream and the bone stream.
The pose prediction based auto-encoder in the self-supervised training stage allows the network to learn motion representation from unlabeled data.
arXiv Detail & Related papers (2022-02-08T16:03:15Z) - Multi-Scale Semantics-Guided Neural Networks for Efficient
Skeleton-Based Human Action Recognition [140.18376685167857]
A simple yet effective multi-scale semantics-guided neural network is proposed for skeleton-based action recognition.
MS-SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
arXiv Detail & Related papers (2021-11-07T03:50:50Z) - SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework
for Person Re-Identification [12.347724797202865]
Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications.
Existing methods typically learn body and motion features from the body-joint trajectory.
We propose a Self-supervised Multi-scale Graph (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics.
arXiv Detail & Related papers (2021-07-05T09:53:08Z) - Multi Scale Temporal Graph Networks For Skeleton-based Action
Recognition [5.970574258839858]
Graph convolutional networks (GCNs) can effectively capture the features of related nodes and improve the performance of the model.
Existing methods based on GCNs have two problems. First, the consistency of temporal and spatial features is ignored for extracting features node by node and frame by frame.
We propose a novel model called Temporal Graph Networks (TGN) for action recognition.
arXiv Detail & Related papers (2020-12-05T08:08:25Z) - Jointly Cross- and Self-Modal Graph Attention Network for Query-Based
Moment Localization [77.21951145754065]
We propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph.
Our CSMGAN is able to effectively capture high-order interactions between two modalities, thus enabling a further precise localization.
arXiv Detail & Related papers (2020-08-04T08:25:24Z) - Unifying Graph Embedding Features with Graph Convolutional Networks for
Skeleton-based Action Recognition [18.001693718043292]
We propose a novel framework, which unifies 15 graph embedding features into the graph convolutional network for human action recognition.
Our model is validated by three large-scale datasets, namely NTU-RGB+D, Kinetics and SYSU-3D.
arXiv Detail & Related papers (2020-03-06T02:31:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.