Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard
Skeleton Mining for Unsupervised Person Re-Identification
- URL: http://arxiv.org/abs/2307.12917v4
- Date: Tue, 19 Sep 2023 01:54:21 GMT
- Title: Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard
Skeleton Mining for Unsupervised Person Re-Identification
- Authors: Haocong Rao, Cyril Leung, Chunyan Miao
- Abstract summary: This paper proposes a generic unsupervised Hierarchical skeleton Meta-Prototype Contrastive learning (Hi-MPC) approach with Hard Skeleton Mining (HSM) for person re-ID with unlabeled 3D skeletons.
By converting original prototypes into meta-prototypes with multiple homogeneous transformations, we induce the model to learn the inherent consistency of prototypes to capture more effective skeleton features for person re-ID.
- Score: 70.90142717649785
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With rapid advancements in depth sensors and deep learning, skeleton-based
person re-identification (re-ID) models have recently achieved remarkable
progress with many advantages. Most existing solutions learn single-level
skeleton features from body joints with the assumption of equal skeleton
importance, while they typically lack the ability to exploit more informative
skeleton features from various levels such as limb level with more global body
patterns. The label dependency of these methods also limits their flexibility
in learning more general skeleton representations. This paper proposes a
generic unsupervised Hierarchical skeleton Meta-Prototype Contrastive learning
(Hi-MPC) approach with Hard Skeleton Mining (HSM) for person re-ID with
unlabeled 3D skeletons. Firstly, we construct hierarchical representations of
skeletons to model coarse-to-fine body and motion features from the levels of
body joints, components, and limbs. Then a hierarchical meta-prototype
contrastive learning model is proposed to cluster and contrast the most typical
skeleton features ("prototypes") from different-level skeletons. By converting
original prototypes into meta-prototypes with multiple homogeneous
transformations, we induce the model to learn the inherent consistency of
prototypes to capture more effective skeleton features for person re-ID.
Furthermore, we devise a hard skeleton mining mechanism to adaptively infer the
informative importance of each skeleton, so as to focus on harder skeletons to
learn more discriminative skeleton representations. Extensive evaluations on
five datasets demonstrate that our approach outperforms a wide variety of
state-of-the-art skeleton-based methods. We further show the general
applicability of our method to cross-view person re-ID and RGB-based scenarios
with estimated skeletons.
Related papers
- SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence
Pre-training [110.55093254677638]
We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL)
In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE.
Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
arXiv Detail & Related papers (2023-07-17T13:33:11Z) - One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton
Matching [77.6989219290789]
One-shot skeleton action recognition aims to learn a skeleton action recognition model with a single training sample.
This paper presents a novel one-shot skeleton action recognition technique that handles skeleton action recognition via multi-scale spatial-temporal feature matching.
arXiv Detail & Related papers (2023-07-14T11:52:10Z) - Skeleton Prototype Contrastive Learning with Multi-Level Graph Relation
Modeling for Unsupervised Person Re-Identification [63.903237777588316]
Person re-identification (re-ID) via 3D skeletons is an important emerging topic with many merits.
Existing solutions rarely explore valuable body-component relations in skeletal structure or motion.
This paper proposes a generic unsupervised Prototype Contrastive learning paradigm with Multi-level Graph Relation learning.
arXiv Detail & Related papers (2022-08-25T00:59:32Z) - Skeleton-based Action Recognition via Adaptive Cross-Form Learning [75.92422282666767]
Skeleton-based action recognition aims to project skeleton sequences to action categories, where sequences are derived from multiple forms of pre-detected points.
Existing methods tend to improve GCNs by leveraging multi-form skeletons due to their complementary cues.
We present Adaptive Cross-Form Learning (ACFL), which empowers well-designed GCNs to generate complementary representation from single-form skeletons.
arXiv Detail & Related papers (2022-06-30T07:40:03Z) - SimMC: Simple Masked Contrastive Learning of Skeleton Representations
for Unsupervised Person Re-Identification [63.903237777588316]
We present a generic Simple Masked Contrastive learning (SimMC) framework to learn effective representations from unlabeled 3D skeletons for person re-ID.
Specifically, to fully exploit skeleton features within each skeleton sequence, we first devise a masked prototype contrastive learning (MPC) scheme.
Then, we propose the masked intra-sequence contrastive learning (MIC) to capture intra-sequence pattern consistency between subsequences.
arXiv Detail & Related papers (2022-04-21T00:19:38Z) - Skeleton-Contrastive 3D Action Representation Learning [35.06361753065124]
This paper strives for self-supervised learning of a feature space suitable for skeleton-based action recognition.
Our approach achieves state-of-the-art performance for self-supervised learning from skeleton data on the challenging PKU and NTU datasets.
arXiv Detail & Related papers (2021-08-08T14:44:59Z) - SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework
for Person Re-Identification [12.347724797202865]
Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications.
Existing methods typically learn body and motion features from the body-joint trajectory.
We propose a Self-supervised Multi-scale Graph (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics.
arXiv Detail & Related papers (2021-07-05T09:53:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.