SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognition

• URL: http://arxiv.org/abs/2603.29692v1
• Date: Tue, 31 Mar 2026 12:46:21 GMT
• Title: SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognition
• Abstract summary: Zero-shot skeleton-based action recognition aims to recognize unseen actions by transferring knowledge from seen categories through semantic descriptions.<n>Most existing methods typically align skeleton features with textual embeddings within a shared latent space.<n>We propose SkeletonContext, a prompt-based framework that enriches skeletal motion representations with language-driven contextual semantics.
• Score: 22.87993245982946
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot skeleton-based action recognition aims to recognize unseen actions by transferring knowledge from seen categories through semantic descriptions. Most existing methods typically align skeleton features with textual embeddings within a shared latent space. However, the absence of contextual cues, such as objects involved in the action, introduces an inherent gap between skeleton and semantic representations, making it difficult to distinguish visually similar actions. To address this, we propose SkeletonContext, a prompt-based framework that enriches skeletal motion representations with language-driven contextual semantics. Specifically, we introduce a Cross-Modal Context Prompt Module, which leverages a pretrained language model to reconstruct masked contextual prompts under guidance derived from LLMs. This design effectively transfers linguistic context to the skeleton encoder for instance-level semantic grounding and improved cross-modal alignment. In addition, a Key-Part Decoupling Module is incorporated to decouple motion-relevant joint features, ensuring robust action understanding even in the absence of explicit object interactions. Extensive experiments on multiple benchmarks demonstrate that SkeletonContext achieves state-of-the-art performance under both conventional and generalized zero-shot settings, validating its effectiveness in reasoning about context and distinguishing fine-grained, visually similar actions.

Related papers

• Novel Semantic Prompting for Zero-Shot Action Recognition [0.0]
  Zero-shot action recognition relies on transferring knowledge from vision-language models to unseen actions using semantic descriptions.<n>We introduce SP-CLIP, a lightweight framework that augments frozen vision-language models with structured semantic prompts describing actions at multiple levels of abstraction.
  arXiv  Detail & Related papers  (2026-03-09T12:07:55Z)
• Beyond Global Alignment: Fine-Grained Motion-Language Retrieval via Pyramidal Shapley-Taylor Learning [56.6025512458557]
  Motion-language retrieval aims to bridge the semantic gap between natural language and human motion.<n>Existing approaches predominantly focus on aligning entire motion sequences with global textual representations.<n>We propose a novel Pyramidal Shapley-Taylor (PST) learning framework for fine-grained motion-language retrieval.
  arXiv  Detail & Related papers  (2026-01-29T16:00:12Z)
• PALUM: Part-based Attention Learning for Unified Motion Retargeting [53.17113525688095]
  Remotion between characters with different skeleton structures is a fundamental challenge in computer animation.<n>We present a novel approach that learns common motion representations across diverse skeleton topologies.<n>Experiments demonstrate superior performance in handling diverse skeletal structures while maintaining motion realism and semantic fidelity.
  arXiv  Detail & Related papers  (2026-01-12T07:29:44Z)
• Including Semantic Information via Word Embeddings for Skeleton-based Action Recognition [7.441242294426765]
  We introduce a novel approach to skeleton-based action recognition that enriches input representations by leveraging word embeddings to encode semantic information.<n>Our method replaces one-hot encodings with semantic volumes, enabling the model to capture meaningful relationships between joints and objects.
  arXiv  Detail & Related papers  (2025-06-23T14:57:06Z)
• Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learning [58.73625654718187]
  Generalized zero-shot learning aims to recognize both seen and unseen classes with the help of semantic information that is shared among different classes.<n>Existing approaches fine-tune the visual backbone by seen-class data to obtain semantic-related visual features.<n>This paper proposes a novel visual and semantic prompt collaboration framework, which utilizes prompt tuning techniques for efficient feature adaptation.
  arXiv  Detail & Related papers  (2025-03-29T10:17:57Z)
• Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning [20.34477942813382]
  Skeleton-based action representation learning aims to interpret and understand human behaviors by encoding the skeleton sequences. We introduce a novel skeleton-based training framework based on Cross-modal Contrastive learning. Our method outperforms the previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-05-31T03:40:15Z)
• Iconic Gesture Semantics [87.00251241246136]
  Informational evaluation is spelled out as extended exemplification (extemplification) in terms of perceptual classification of a gesture's visual iconic model. We argue that the perceptual classification of instances of visual communication requires a notion of meaning different from Frege/Montague frameworks. An iconic gesture semantics is introduced which covers the full range from gesture representations over model-theoretic evaluation to inferential interpretation in dynamic semantic frameworks.
  arXiv  Detail & Related papers  (2024-04-29T13:58:03Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• Contextual Modulation for Relation-Level Metaphor Identification [3.2619536457181075]
  We introduce a novel architecture for identifying relation-level metaphoric expressions of certain grammatical relations. In a methodology inspired by works in visual reasoning, our approach is based on conditioning the neural network computation on the deep contextualised features. We demonstrate that the proposed architecture achieves state-of-the-art results on benchmark datasets.
  arXiv  Detail & Related papers  (2020-10-12T12:07:02Z)
• Skeleton-Aware Networks for Deep Motion Retargeting [83.65593033474384]
  We introduce a novel deep learning framework for data-driven motion between skeletons. Our approach learns how to retarget without requiring any explicit pairing between the motions in the training set.
  arXiv  Detail & Related papers  (2020-05-12T12:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.