Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions

• URL: http://arxiv.org/abs/2605.22697v1
• Date: Thu, 21 May 2026 16:39:58 GMT
• Title: Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions
• Abstract summary: This paper presents a novel orientation-aware action recognition approach with improved cross-domain capabilities.<n>We present a new orientation-aware motion encoding network to learn different motion features, and adapt a specific orientation-aware text prompt to match the corresponding features at inference.
• Score: 2.0019258650495493
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Robustness to domain changes is a key capability for effective deployment of human action recognition systems in real-world scenarios, where action categories at inference can present important domain shifts or even unseen actions from training. In this context, improving the recognition capabilities of Zero-Shot Action Recognition models (ZSAR), without requiring strong annotation efforts, remains a central challenge. Most ZSAR approaches assume that actions are observed under geometric conditions similar to those seen during training. In practice, variations in human body orientation and camera viewpoint add a significant domain gap in ZSAR, substantially limiting generalization to novel action-motion combinations. In this context, this paper presents a novel orientation-aware action recognition approach with improved cross-domain capabilities. Our approach combines motion cues of multiple camera viewpoints and text descriptions of human actions in the training phase. We present a new orientation-aware motion encoding network to learn different motion features, and adapt a specific orientation-aware text prompt to match the corresponding features at inference. Extensive experiments demonstrate that the proposed method consistently improves ZSAR performance across different recognition benchmarks, outperforming recent state-of-the-art zero-shot approaches on NTU-RGB+D, BABEL, NW-UCLA, and on two surveillance datasets. In addition, the learned representations exhibit strong transfer learning capabilities, yielding competitive performance on both cross-domain and same-domain recognition of seen actions. Code and trained models are available at: https://icb-vision-ai.github.io/OrientationAware-HAR

Related papers

• DiSa: Directional Saliency-Aware Prompt Learning for Generalizable Vision-Language Models [5.027492394254859]
  DiSa is a Directional Saliency-Aware Prompt Learning framework.<n>It integrates two complementary regularization strategies to enhance generalization.<n>It consistently outperforms state-of-the-art prompt learning methods across various settings.
  arXiv  Detail & Related papers  (2025-05-26T00:14:52Z)
• SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrations [68.9300049150948]
  We address a fundamental challenge in Reinforcement Learning from Interaction Demonstration (RLID)<n>Existing data collection approaches yield sparse, disconnected, and noisy trajectories that fail to capture the full spectrum of possible skill variations and transitions.<n>We present two data augmentation techniques: a Stitched Trajectory Graph (STG) that discovers potential transitions between demonstration skills, and a State Transition Field (STF) that establishes unique connections for arbitrary states within the demonstration neighborhood.
  arXiv  Detail & Related papers  (2025-05-04T13:00:29Z)
• Boosting Single-domain Generalized Object Detection via Vision-Language Knowledge Interaction [4.692621855184482]
  Single-Domain Generalized Object Detection(S-DGOD) aims to train an object detector on a single source domain.<n>Recent S-DGOD approaches exploit pre-trained vision-language knowledge to guide invariant feature learning across visual domains.<n>We propose a new cross-modal feature learning method, which can capture generalized and discriminative regional features for S-DGOD tasks.
  arXiv  Detail & Related papers  (2025-04-27T02:55:54Z)
• Adversarial Domain Adaptation for Action Recognition Around the Clock [0.7614628596146599]
  This paper presents a domain adaptation-based action recognition approach. It uses adversarial learning in cross-domain settings to learn cross-domain action recognition. It achieves SOTA performance on InFAR and XD145 actions datasets.
  arXiv  Detail & Related papers  (2022-10-25T01:08:27Z)
• Audio-Adaptive Activity Recognition Across Video Domains [112.46638682143065]
  We leverage activity sounds for domain adaptation as they have less variance across domains and can reliably indicate which activities are not happening. We propose an audio-adaptive encoder and associated learning methods that discriminatively adjust the visual feature representation. We also introduce the new task of actor shift, with a corresponding audio-visual dataset, to challenge our method with situations where the activity appearance changes dramatically.
  arXiv  Detail & Related papers  (2022-03-27T08:15:20Z)
• Domain-Aware Continual Zero-Shot Learning [52.349332188116975]
  Domain-Aware Continual Zero-Shot Learning (DACZSL) is a task to recognize images of unseen categories in continuously changing domains. We propose a Domain-Invariant Network (DIN) to learn factorized features for shifting domains and improved textual representation for unseen classes. Our results show that DIN significantly outperforms existing baselines by over 5% in harmonic accuracy and over 1% in backward transfer.
  arXiv  Detail & Related papers  (2021-12-24T08:17:18Z)
• Efficient Modelling Across Time of Human Actions and Interactions [92.39082696657874]
  We argue that current fixed-sized-temporal kernels in 3 convolutional neural networks (CNNDs) can be improved to better deal with temporal variations in the input. We study how we can better handle between classes of actions, by enhancing their feature differences over different layers of the architecture. The proposed approaches are evaluated on several benchmark action recognition datasets and show competitive results.
  arXiv  Detail & Related papers  (2021-10-05T15:39:11Z)
• Learning Cross-modal Contrastive Features for Video Domain Adaptation [138.75196499580804]
  We propose a unified framework for video domain adaptation, which simultaneously regularizes cross-modal and cross-domain feature representations. Specifically, we treat each modality in a domain as a view and leverage the contrastive learning technique with properly designed sampling strategies.
  arXiv  Detail & Related papers  (2021-08-26T18:14:18Z)
• Elaborative Rehearsal for Zero-shot Action Recognition [36.84404523161848]
  ZSAR aims to recognize target (unseen) actions without training examples. It remains challenging to semantically represent action classes and transfer knowledge from seen data. We propose an ER-enhanced ZSAR model inspired by an effective human memory technique Elaborative Rehearsal.
  arXiv  Detail & Related papers  (2021-08-05T20:02:46Z)
• Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment [60.31418655784291]
  We propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues in videos, and inherent correlations in multi-modal towards recognizing gesture. Results show that our approach recovers the performance with great improvement gains, up to 12.91% in ACC and 20.16% in F1score without using any annotations in real robot.
  arXiv  Detail & Related papers  (2021-03-06T09:10:03Z)
• Domain and View-point Agnostic Hand Action Recognition [6.432798111887824]
  We introduce a novel skeleton-based hand motion representation model that tackles this problem. We demonstrate the performance of our proposed motion representation model both working for a single specific domain (intra-domain action classification) and working for different unseen domains (cross-domain action classification) Our approach achieves comparable results to the state-of-the-art methods that are trained intra-domain.
  arXiv  Detail & Related papers  (2021-03-03T10:32:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.