Point-Supervised Skeleton-Based Human Action Segmentation

• URL: http://arxiv.org/abs/2603.06201v1
• Date: Fri, 06 Mar 2026 12:14:50 GMT
• Title: Point-Supervised Skeleton-Based Human Action Segmentation
• Abstract summary: We introduce a point-supervised framework for skeleton-based action segmentation, where only a single frame per action segment is labeled.<n>We leverage multimodal skeleton data, including joint, bone, and motion information, encoded via a pretrained unified model to extract rich feature representations.<n>Our method achieves competitive performance, even surpassing some fully-supervised methods.
• Score: 23.335246964537223
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Skeleton-based temporal action segmentation is a fundamental yet challenging task, playing a crucial role in enabling intelligent systems to perceive and respond to human activities. While fully-supervised methods achieve satisfactory performance, they require costly frame-level annotations and are sensitive to ambiguous action boundaries. To address these issues, we introduce a point-supervised framework for skeleton-based action segmentation, where only a single frame per action segment is labeled. We leverage multimodal skeleton data, including joint, bone, and motion information, encoded via a pretrained unified model to extract rich feature representations. To generate reliable pseudo-labels, we propose a novel prototype similarity method and integrate it with two existing methods: energy function and constrained K-Medoids clustering. Multimodal pseudo-label integration is proposed to enhance the reliability of the pseudo-label and guide the model training. We establish new benchmarks on PKU-MMD (X-Sub and X-View), MCFS-22, and MCFS-130, and implement baselines for point-supervised skeleton-based human action segmentation. Extensive experiments show that our method achieves competitive performance, even surpassing some fully-supervised methods while significantly reducing annotation effort.

Related papers

• SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition [54.97159515886273]
  SkelHCC is a unified skeleton hyperbolic CLIP-driven cache adaptation framework for one-shot action recognition.<n>By leveraging the negative curvature and exponential volume growth of hyperbolic geometry, EH-HCLIP naturally encodes the joint-part-body hierarchy of human anatomy.
  arXiv  Detail & Related papers  (2026-06-02T13:13:27Z)
• Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samples [51.59753385094941]
  3D Action Recognition with Limited Training Samples, also known as semi-supervised 3D Action Recognition, has been proposed.<n>We reformulate Semi-supervised 3D action recognition via active learning from a novel perspective by casting it as a Markov Decision Process (MDP)<n>To enhance the representational capacity of the factors in the state-action pairs within our method, we project them from Euclidean space to hyperbolic space.
  arXiv  Detail & Related papers  (2025-10-29T10:03:33Z)
• Foundation Model for Skeleton-Based Human Action Understanding [56.89025287217221]
  This paper presents a Unified Skeleton-based Dense Representation Learning framework.<n>USDRL consists of a Transformer-based Dense Spatio-Temporal (DSTE), Multi-Grained Feature Decorrelation (MG-FD), and Multi-Perspective Consistency Training (MPCT)
  arXiv  Detail & Related papers  (2025-08-18T02:42:16Z)
• Towards Open-World Human Action Segmentation Using Graph Convolutional Networks [6.167678490008973]
  Most existing learning-based methods excel in closed-world action segmentation.<n>We propose a structured framework for detecting and segmenting unseen actions.<n>We evaluate our framework on two challenging human-object recognition datasets.
  arXiv  Detail & Related papers  (2025-07-01T14:00:39Z)
• An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition [49.45660055499103]
  Zero-shot human skeleton-based action recognition aims to construct a model that can recognize actions outside the categories seen during training. Previous research has focused on aligning sequences' visual and semantic spatial distributions. We introduce a new loss function sampling method to obtain a tight and robust representation.
  arXiv  Detail & Related papers  (2024-06-02T06:53:01Z)
• Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences [29.376328807860993]
  We propose a Partial Spatio-Temporal Learning (PSTL) framework to exploit the local relationship between different skeleton joints and video frames. Our method achieves state-of-the-art performance on NTURGB+D 60, NTURGBMM+D 120 and PKU-D under various downstream tasks.
  arXiv  Detail & Related papers  (2023-02-17T17:35:05Z)
• Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning [146.11600461034746]
  Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
  arXiv  Detail & Related papers  (2022-09-27T19:04:36Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition [65.78703941973183]
  We propose a novel correlation-driven joint-bone fusion graph convolutional network (CD-JBF-GCN) as an encoder and use a pose prediction head as a decoder. Specifically, the CD-JBF-GC can explore the motion transmission between the joint stream and the bone stream. The pose prediction based auto-encoder in the self-supervised training stage allows the network to learn motion representation from unlabeled data.
  arXiv  Detail & Related papers  (2022-02-08T16:03:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.