Frequency-Enhanced Diffusion Models: Curriculum-Guided Semantic Alignment for Zero-Shot Skeleton Action Recognition

• URL: http://arxiv.org/abs/2604.09063v1
• Date: Fri, 10 Apr 2026 07:42:47 GMT
• Title: Frequency-Enhanced Diffusion Models: Curriculum-Guided Semantic Alignment for Zero-Shot Skeleton Action Recognition
• Abstract summary: We propose Frequency-Aware Diffusion for Skeleton-Text Matching (FDSM)<n>Our approach effectively recovers fine-grained motion details, achieving state-of-the-art performance on NTU RGB+D, PKU-MMD, and Kinetics-skeleton datasets.
• Score: 13.27335417467632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human action recognition is pivotal in computer vision, with applications ranging from surveillance to human-robot interaction. Despite the effectiveness of supervised skeleton-based methods, their reliance on exhaustive annotation limits generalization to novel actions. Zero-Shot Skeleton Action Recognition (ZSAR) emerges as a promising paradigm, yet it faces challenges due to the spectral bias of diffusion models, which oversmooth high-frequency dynamics. Here, we propose Frequency-Aware Diffusion for Skeleton-Text Matching (FDSM), integrating a Semantic-Guided Spectral Residual Module, a Timestep-Adaptive Spectral Loss, and Curriculum-based Semantic Abstraction to address these challenges. Our approach effectively recovers fine-grained motion details, achieving state-of-the-art performance on NTU RGB+D, PKU-MMD, and Kinetics-skeleton datasets. Code has been made available at https://github.com/yuzhi535/FDSM. Project homepage: https://yuzhi535.github.io/FDSM.github.io/

Related papers

• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• SkeFi: Cross-Modal Knowledge Transfer for Wireless Skeleton-Based Action Recognition [20.020503149009787]
  Existing solutions use RGB cameras to annotate skeletal keypoints, but their performance declines in dark environments and raises privacy concerns.<n>This paper explores non-invasive wireless sensors, i.e., LiDAR and mmWave, to mitigate these challenges.<n> Experiments demonstrate that SkeFi realizes state-of-the-art performances on mmWave and LiDAR.
  arXiv  Detail & Related papers  (2026-01-18T14:39:02Z)
• DynaPURLS: Dynamic Refinement of Part-aware Representations for Skeleton-based Zero-Shot Action Recognition [51.80782323686666]
  We introduce textbfDynaPURLS, a unified framework that establishes robust, multi-scale visual-semantic correspondences.<n>Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics.<n>Experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art.
  arXiv  Detail & Related papers  (2025-12-12T10:39:10Z)
• Frequency-Semantic Enhanced Variational Autoencoder for Zero-Shot Skeleton-based Action Recognition [11.11236920942621]
  Zero-shot skeleton-based action recognition aims to identify actions beyond the categories encountered during training.<n>Previous approaches have primarily focused on aligning visual and semantic representations.<n>We propose a Frequency-Semantic Enhanced Variational Autoencoder (FS-VAE) to explore the skeleton semantic representation learning with frequency decomposition.
  arXiv  Detail & Related papers  (2025-06-27T12:44:08Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer [18.459822172890473]
  We introduce a frequency-aware attention module to unweave skeleton frequency representations. We also develop a mixed transformer architecture to incorporate spatial features with frequency features. Experiments show that FreqMiXFormer outperforms SOTA on 3 popular skeleton recognition datasets.
  arXiv  Detail & Related papers  (2024-07-17T05:47:27Z)
• An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition [49.45660055499103]
  Zero-shot human skeleton-based action recognition aims to construct a model that can recognize actions outside the categories seen during training. Previous research has focused on aligning sequences' visual and semantic spatial distributions. We introduce a new loss function sampling method to obtain a tight and robust representation.
  arXiv  Detail & Related papers  (2024-06-02T06:53:01Z)
• Shape Conditioned Human Motion Generation with Diffusion Model [0.0]
  We propose a Shape-conditioned Motion Diffusion model (SMD), which enables the generation of motion sequences directly in mesh format. We also propose a Spectral-Temporal Autoencoder (STAE) to leverage cross-temporal dependencies within the spectral domain.
  arXiv  Detail & Related papers  (2024-05-10T19:06:41Z)
• SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognition [13.283178393519234]
  Self-supervised skeleton-based action recognition has attracted more attention. With utilizing the unlabeled data, more generalizable features can be learned to alleviate the overfitting problem. We propose a spatial-temporal masked autoencoder framework for self-supervised 3D skeleton-based action recognition.
  arXiv  Detail & Related papers  (2022-09-01T20:54:27Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Learn to cycle: Time-consistent feature discovery for action recognition [83.43682368129072]
  Generalizing over temporal variations is a prerequisite for effective action recognition in videos. We introduce Squeeze Re Temporal Gates (SRTG), an approach that favors temporal activations with potential variations. We show consistent improvement when using SRTPG blocks, with only a minimal increase in the number of GFLOs.
  arXiv  Detail & Related papers  (2020-06-15T09:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.