Decompose and Transfer: CoT-Prompting Enhanced Alignment for Open-Vocabulary Temporal Action Detection

• URL: http://arxiv.org/abs/2603.24030v1
• Date: Wed, 25 Mar 2026 07:45:04 GMT
• Title: Decompose and Transfer: CoT-Prompting Enhanced Alignment for Open-Vocabulary Temporal Action Detection
• Abstract summary: Open-Vocabulary Temporal Action Detection aims to classify and localize action segments in untrimmed videos for unseen categories.<n>Previous methods rely solely on global alignment between label-level semantics and visual features.<n>We propose a Phase-wise Decomposition and Alignment framework, which enables fine-grained action pattern learning.
• Score: 21.20482802952099
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-Vocabulary Temporal Action Detection (OV-TAD) aims to classify and localize action segments in untrimmed videos for unseen categories. Previous methods rely solely on global alignment between label-level semantics and visual features, which is insufficient to transfer temporal consistent visual knowledge from seen to unseen classes. To address this, we propose a Phase-wise Decomposition and Alignment (PDA) framework, which enables fine-grained action pattern learning for effective prior knowledge transfer. Specifically, we first introduce the CoT-Prompting Semantic Decomposition (CSD) module, which leverages the chain-of-thought (CoT) reasoning ability of large language models to automatically decompose action labels into coherent phase-level descriptions, emulating human cognitive processes. Then, Text-infused Foreground Filtering (TIF) module is introduced to adaptively filter action-relevant segments for each phase leveraging phase-wise semantic cues, producing semantically aligned visual representations. Furthermore, we propose the Adaptive Phase-wise Alignment (APA) module to perform phase-level visual-textual matching, and adaptively aggregates alignment results across phases for final prediction. This adaptive phase-wise alignment facilitates the capture of transferable action patterns and significantly enhances generalization to unseen actions. Extensive experiments on two OV-TAD benchmarks demonstrated the superiority of the proposed method.

Related papers

• Learning a Semantic Calibration Network for Open-Vocabulary Semantic Segmentation [5.122331812021513]
  Open-Vocabulary (OVS) extends the segmentation task from a fixed set to an open set.<n>We propose a novel Semantic Network (SCN) for open-vocabulary semantic segmentation.<n>The proposed method achieves significant performance improvements compared to state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2026-06-06T06:42:03Z)
• PRISM: Iterative Cross-Modal Posterior Refinement for Dynamic Text-Attributed Graphs [5.2307029116231165]
  multimodal learning has emerged as a promising yet underexplored direction for enhancing DyTAG representation learning.<n>We propose textbfPRISM, an iterative cross-modal posterior refinement framework for DyTAG representation learning.<n>PRISM organizes DyTAG information into semantic and behavioral modalities, providing a more intrinsic alternative to carrier-level modality partitions.
  arXiv  Detail & Related papers  (2026-05-07T11:58:47Z)
• AttriPrompt: Dynamic Prompt Composition Learning for CLIP [41.37140060183439]
  AttriPrompt is a novel framework that enhances and refines textual semantic representations.<n>We introduce a Self-Regularization mechanism by applying explicit regularization constraints between the prompted and non-prompted text features.<n>Experiments demonstrate AttriPrompt's superiority over state-of-the-art methods, achieving up to 7.37% improvement in the base-to-novel setting.
  arXiv  Detail & Related papers  (2025-09-07T07:07:59Z)
• Plug-in Feedback Self-adaptive Attention in CLIP for Training-free Open-Vocabulary Segmentation [48.488114831677166]
  CLIP exhibits strong visual-textual alignment but struggle with open-vocabulary segmentation due to poor localization.<n>We propose a training-free, feedback-driven self-adaptive framework that adapts output-based patch-level correspondences back to the intermediate attention.
  arXiv  Detail & Related papers  (2025-08-27T20:47:03Z)
• Training-Free Class Purification for Open-Vocabulary Semantic Segmentation [72.87707878910896]
  FreeCP is a training-free class purification framework for semantic segmentation.<n>We conduct experiments across eight benchmarks to validate FreeCP's effectiveness.<n>Results demonstrate that FreeCP, as a plug-and-play module, significantly boosts segmentation performance when combined with other OVSS methods.
  arXiv  Detail & Related papers  (2025-08-01T11:55:12Z)
• HAMLET-FFD: Hierarchical Adaptive Multi-modal Learning Embeddings Transformation for Face Forgery Detection [6.060036926093259]
  HAMLET-FFD is a cross-domain generalization framework for face forgery detection.<n>It integrates visual evidence with conceptual cues, emulating expert forensic analysis.<n>By design, HAMLET-FFD freezes all pretrained parameters, serving as an external plugin.
  arXiv  Detail & Related papers  (2025-07-28T15:09:52Z)
• DiSa: Directional Saliency-Aware Prompt Learning for Generalizable Vision-Language Models [5.027492394254859]
  DiSa is a Directional Saliency-Aware Prompt Learning framework.<n>It integrates two complementary regularization strategies to enhance generalization.<n>It consistently outperforms state-of-the-art prompt learning methods across various settings.
  arXiv  Detail & Related papers  (2025-05-26T00:14:52Z)
• Semantic-Aligned Learning with Collaborative Refinement for Unsupervised VI-ReID [82.12123628480371]
  Unsupervised person re-identification (USL-VI-ReID) seeks to match pedestrian images of the same individual across different modalities without human annotations for model learning.<n>Previous methods unify pseudo-labels of cross-modality images through label association algorithms and then design contrastive learning framework for global feature learning.<n>We propose a Semantic-Aligned Learning with Collaborative Refinement (SALCR) framework, which builds up objective for specific fine-grained patterns emphasized by each modality.
  arXiv  Detail & Related papers  (2025-04-27T13:58:12Z)
• CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofing [66.6712018832575]
  Domain generalization (DG) based Face Anti-Spoofing (FAS) aims to improve the model's performance on unseen domains. We make use of large-scale VLMs like CLIP and leverage the textual feature to dynamically adjust the classifier's weights for exploring generalizable visual features.
  arXiv  Detail & Related papers  (2024-03-21T11:58:50Z)
• Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localization [87.47977407022492]
  This paper argues that learning by contextually comparing sequence-to-sequence distinctions offers an essential inductive bias in weakly-supervised action localization. Under a differentiable dynamic programming formulation, two complementary contrastive objectives are designed, including Fine-grained Sequence Distance (FSD) contrasting and Longest Common Subsequence (LCS) contrasting. Our method achieves state-of-the-art performance on two popular benchmarks.
  arXiv  Detail & Related papers  (2022-03-31T05:13:50Z)
• Referring Image Segmentation via Cross-Modal Progressive Comprehension [94.70482302324704]
  Referring image segmentation aims at segmenting the foreground masks of the entities that can well match the description given in the natural language expression. Previous approaches tackle this problem using implicit feature interaction and fusion between visual and linguistic modalities. We propose a Cross-Modal Progressive (CMPC) module and a Text-Guided Feature Exchange (TGFE) module to effectively address the challenging task.
  arXiv  Detail & Related papers  (2020-10-01T16:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.