Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation

• URL: http://arxiv.org/abs/2508.04513v1
• Date: Wed, 06 Aug 2025 14:56:38 GMT
• Title: Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation
• Authors: Uzay Gökay, Federico Spurio, Dominik R. Bach, Juergen Gall,
• Abstract summary: We propose a novel approach for unsupervised skeleton-based temporal action segmentation.<n>Our method utilizes a sequence-to-sequence temporal autoencoder that keeps the information of the different joints disentangled in the embedding space.<n>We thoroughly evaluate the proposed approach on three widely used skeleton-based datasets.
• Score: 11.045126693185377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current state-of-the-art methods for skeleton-based temporal action segmentation are predominantly supervised and require annotated data, which is expensive to collect. In contrast, existing unsupervised temporal action segmentation methods have focused primarily on video data, while skeleton sequences remain underexplored, despite their relevance to real-world applications, robustness, and privacy-preserving nature. In this paper, we propose a novel approach for unsupervised skeleton-based temporal action segmentation. Our method utilizes a sequence-to-sequence temporal autoencoder that keeps the information of the different joints disentangled in the embedding space. Latent skeleton sequences are then divided into non-overlapping patches and quantized to obtain distinctive skeleton motion words, driving the discovery of semantically meaningful action clusters. We thoroughly evaluate the proposed approach on three widely used skeleton-based datasets, namely HuGaDB, LARa, and BABEL. The results demonstrate that our model outperforms the current state-of-the-art unsupervised temporal action segmentation methods. Code is available at https://github.com/bachlab/SMQ .

Related papers

• Stitch Contrast and Segment_Learning a Human Action Segmentation Model Using Trimmed Skeleton Videos [3.069335774032178]
  This paper presents a novel framework for skeleton-based action segmentation trained on short trimmed skeleton videos.<n>It is implemented in three steps: Stitch, Contrast, and Segment.<n>Experiments involve a trimmed source dataset and an untrimmed target dataset.
  arXiv  Detail & Related papers  (2024-12-19T16:00:10Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments [40.322770236718775]
  We propose IosPSTL, a simple and effective self-supervised learning framework designed to handle occlusions.<n>IosPSTL combines a cluster-agnostic KNN imputer with an Occluded Partial Spatio-Temporal Learning (OPSTL) strategy.<n>OPSTL module incorporates Adaptive Spatial Masking (ASM) to make better use of intact, high-quality skeleton sequences during training.
  arXiv  Detail & Related papers  (2023-09-21T12:51:11Z)
• SkeleTR: Towrads Skeleton-based Action Recognition in the Wild [86.03082891242698]
  SkeleTR is a new framework for skeleton-based action recognition. It first models the intra-person skeleton dynamics for each skeleton sequence with graph convolutions. It then uses stacked Transformer encoders to capture person interactions that are important for action recognition in general scenarios.
  arXiv  Detail & Related papers  (2023-09-20T16:22:33Z)
• One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching [77.6989219290789]
  One-shot skeleton action recognition aims to learn a skeleton action recognition model with a single training sample. This paper presents a novel one-shot skeleton action recognition technique that handles skeleton action recognition via multi-scale spatial-temporal feature matching.
  arXiv  Detail & Related papers  (2023-07-14T11:52:10Z)
• Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences [29.376328807860993]
  We propose a Partial Spatio-Temporal Learning (PSTL) framework to exploit the local relationship between different skeleton joints and video frames. Our method achieves state-of-the-art performance on NTURGB+D 60, NTURGBMM+D 120 and PKU-D under various downstream tasks.
  arXiv  Detail & Related papers  (2023-02-17T17:35:05Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• SimMC: Simple Masked Contrastive Learning of Skeleton Representations for Unsupervised Person Re-Identification [63.903237777588316]
  We present a generic Simple Masked Contrastive learning (SimMC) framework to learn effective representations from unlabeled 3D skeletons for person re-ID. Specifically, to fully exploit skeleton features within each skeleton sequence, we first devise a masked prototype contrastive learning (MPC) scheme. Then, we propose the masked intra-sequence contrastive learning (MIC) to capture intra-sequence pattern consistency between subsequences.
  arXiv  Detail & Related papers  (2022-04-21T00:19:38Z)
• Sequential convolutional network for behavioral pattern extraction in gait recognition [0.7874708385247353]
  We propose a sequential convolutional network (SCN) to learn the walking pattern of individuals. In SCN, behavioral information extractors (BIE) are constructed to comprehend intermediate feature maps in time series. A multi-frame aggregator in SCN performs feature integration on a sequence whose length is uncertain, via a mobile 3D convolutional layer.
  arXiv  Detail & Related papers  (2021-04-23T08:44:10Z)
• Skeleton-Aware Networks for Deep Motion Retargeting [83.65593033474384]
  We introduce a novel deep learning framework for data-driven motion between skeletons. Our approach learns how to retarget without requiring any explicit pairing between the motions in the training set.
  arXiv  Detail & Related papers  (2020-05-12T12:51:40Z)
• Skeleton Based Action Recognition using a Stacked Denoising Autoencoder with Constraints of Privileged Information [5.67220249825603]
  We propose a new method to study the skeletal representation in a view of skeleton reconstruction. Based on the concept of learning under privileged information, we integrate action categories and temporal coordinates into a stacked denoising autoencoder. In order to mitigate the variation resulting from temporary misalignment, a new method of temporal registration is proposed.
  arXiv  Detail & Related papers  (2020-03-12T09:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.