Contrastive Self-Supervised Learning for Skeleton Representations

• URL: http://arxiv.org/abs/2211.05304v1
• Date: Thu, 10 Nov 2022 02:45:36 GMT
• Title: Contrastive Self-Supervised Learning for Skeleton Representations
• Authors: Nico Lingg, Miguel Sarabia, Luca Zappella and Barry-John Theobald
• Abstract summary: We use a contrastive self-supervised learning method, SimCLR, to learn representations that capture the semantics of skeleton point clouds. To pre-train the representations, we normalise six existing datasets to obtain more than 40 million skeleton frames. We evaluate the quality of the learned representations with three downstream tasks: skeleton reconstruction, motion prediction, and activity classification.
• Score: 2.528877542605869
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human skeleton point clouds are commonly used to automatically classify and predict the behaviour of others. In this paper, we use a contrastive self-supervised learning method, SimCLR, to learn representations that capture the semantics of skeleton point clouds. This work focuses on systematically evaluating the effects that different algorithmic decisions (including augmentations, dataset partitioning and backbone architecture) have on the learned skeleton representations. To pre-train the representations, we normalise six existing datasets to obtain more than 40 million skeleton frames. We evaluate the quality of the learned representations with three downstream tasks: skeleton reconstruction, motion prediction, and activity classification. Our results demonstrate the importance of 1) combining spatial and temporal augmentations, 2) including additional datasets for encoder training, and 3) and using a graph neural network as an encoder.

Related papers

• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training [110.55093254677638]
  We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL) In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE. Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
  arXiv  Detail & Related papers  (2023-07-17T13:33:11Z)
• Self-Supervised 3D Action Representation Learning with Skeleton Cloud Colorization [75.0912240667375]
  3D Skeleton-based human action recognition has attracted increasing attention in recent years. Most of the existing work focuses on supervised learning which requires a large number of labeled action sequences. In this paper, we address self-supervised 3D action representation learning for skeleton-based action recognition.
  arXiv  Detail & Related papers  (2023-04-18T08:03:26Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Skeleton-Contrastive 3D Action Representation Learning [35.06361753065124]
  This paper strives for self-supervised learning of a feature space suitable for skeleton-based action recognition. Our approach achieves state-of-the-art performance for self-supervised learning from skeleton data on the challenging PKU and NTU datasets.
  arXiv  Detail & Related papers  (2021-08-08T14:44:59Z)
• Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning [65.88887113157627]
  Skeleton-based human action recognition has attracted increasing attention in recent years. We design a novel skeleton cloud colorization technique that is capable of learning skeleton representations from unlabeled skeleton sequence data. We show that the proposed method outperforms existing unsupervised and semi-supervised 3D action recognition methods by large margins.
  arXiv  Detail & Related papers  (2021-08-04T10:55:39Z)
• A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification [65.18004601366066]
  Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID.
  arXiv  Detail & Related papers  (2020-09-05T16:06:04Z)
• Unifying Graph Embedding Features with Graph Convolutional Networks for Skeleton-based Action Recognition [18.001693718043292]
  We propose a novel framework, which unifies 15 graph embedding features into the graph convolutional network for human action recognition. Our model is validated by three large-scale datasets, namely NTU-RGB+D, Kinetics and SYSU-3D.
  arXiv  Detail & Related papers  (2020-03-06T02:31:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.