Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining

• URL: http://arxiv.org/abs/2608.25701v1
• Date: Wed, 26 Aug 2026 12:17:07 GMT
• Title: Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining
• Abstract summary: We propose a pretraining strategy for zero-shot-supervised skeleton-based action localization to estimate unseen actions for person instances.<n>We also propose a Scene-Mixed Discriminative Contrastive Learning to distinguish actions at the instance level within the combined scene.
• Score: 11.621071097578808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel pretraining strategy for skeleton-based zero-shot spatio-temporal action localization to estimate unseen actions for person instances while overcoming high annotation costs for training via new target actions and pretraining using large-scale action scenery datasets. Specifically, our approach, termed Skeleton-Language feature Pooling Switching, introduces a weakly-supervised vision-language pretraining mechanism. This mechanism transitions pooling kernels from pretraining, which aggregates skeleton features at the video level and aligns them with each video's known action text embeddings, to the inference phase that computes instance-level features without training via target actions. Furthermore, we propose Scene-Mixed Discriminative Contrastive Learning to distinguish actions at the instance level within the combined scene through the MIL framework. Our experiments on four public spatio-temporal action localization and classification datasets demonstrate that the proposed method effectively addresses annotation limitations.

Related papers

• TF-CADE: Foreground-Concentrated Text-Video Alignment for Zero-Shot Temporal Action Detection [54.38725298163703]
  We propose a Text-Foreground Concentrated Alignment for zero-shot temporal action DEtector (TF-CADE)<n>ACA extracts action concentrate scores to aggregate temporally informative video segments into a foreground- weighted video embedding.<n> CCR strategy refines per-snippet confidence scores by lever- aging foreground-aware similarity, effectively suppressing irrelevant action classes during inference.
  arXiv  Detail & Related papers  (2026-08-18T06:43:45Z)
• Skeleton-Snippet Contrastive Learning with Multiscale Feature Fusion for Action Localization [8.574131591092138]
  We develop a snippet discrimination pretext task for self-supervised pretraining.<n>We also build on strong backbones of skeleton-based action recognition models by fusing intermediate features with a U-shaped module.<n>Our approach consistently improves existing skeleton-based contrastive learning methods for action localization on BABEL.
  arXiv  Detail & Related papers  (2025-12-18T13:15:52Z)
• Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samples [51.59753385094941]
  3D Action Recognition with Limited Training Samples, also known as semi-supervised 3D Action Recognition, has been proposed.<n>We reformulate Semi-supervised 3D action recognition via active learning from a novel perspective by casting it as a Markov Decision Process (MDP)<n>To enhance the representational capacity of the factors in the state-action pairs within our method, we project them from Euclidean space to hyperbolic space.
  arXiv  Detail & Related papers  (2025-10-29T10:03:33Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• Multi-modal Prompting for Low-Shot Temporal Action Localization [95.19505874963751]
  We consider the problem of temporal action localization under low-shot (zero-shot & few-shot) scenario. We adopt a Transformer-based two-stage action localization architecture with class-agnostic action proposal, followed by open-vocabulary classification.
  arXiv  Detail & Related papers  (2023-03-21T10:40:13Z)
• Action Shuffling for Weakly Supervised Temporal Localization [22.43209053892713]
  This paper analyzes the order-sensitive and location-insensitive properties of actions. It embodies them into a self-augmented learning framework to improve the weakly supervised action localization performance.
  arXiv  Detail & Related papers  (2021-05-10T09:05:58Z)
• Weakly Supervised Temporal Action Localization Through Learning Explicit Subspaces for Action and Context [151.23835595907596]
  Methods learn to localize temporal starts and ends of action instances in a video under only video-level supervision. We introduce a framework that learns two feature subspaces respectively for actions and their context. The proposed approach outperforms state-of-the-art WS-TAL methods on three benchmarks.
  arXiv  Detail & Related papers  (2021-03-30T08:26:53Z)
• Point-Level Temporal Action Localization: Bridging Fully-supervised Proposals to Weakly-supervised Losses [84.2964408497058]
  Point-level temporal action localization (PTAL) aims to localize actions in untrimmed videos with only one timestamp annotation for each action instance. Existing methods adopt the frame-level prediction paradigm to learn from the sparse single-frame labels. This paper attempts to explore the proposal-based prediction paradigm for point-level annotations.
  arXiv  Detail & Related papers  (2020-12-15T12:11:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.