LEViL: Label-Efficient Video Learning via Zero-Shot Distillation over VLM-Generated Pseudo-Label Spaces

• URL: http://arxiv.org/abs/2606.21358v1
• Date: Fri, 19 Jun 2026 12:01:10 GMT
• Title: LEViL: Label-Efficient Video Learning via Zero-Shot Distillation over VLM-Generated Pseudo-Label Spaces
• Abstract summary: Supervised video pretraining is a common transfer learning practice for improving downstream action recognition performance.<n>This study proposes a label-efficient video learning framework that combines annotation-free video pretraining with target-label-set-aware fine-tuning.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Supervised video pretraining is a common transfer learning practice for improving downstream action recognition performance. However, it requires large-scale labeled source datasets, and the effectiveness of the learned initialization is influenced by the similarity between the source and target domains. Constructing such labeled pretraining datasets for different target domains is costly and difficult to scale. To address these limitations, this study proposes a label-efficient video learning framework that combines annotation-free video pretraining with target-label-set-aware fine-tuning. During pretraining, a vision-language model (VLM) generates textual descriptions of unlabeled videos, which are processed to construct an interpretable semantic pseudo-label space. A frozen video-language model then produces zero-shot soft target distributions over this space, allowing a student video encoder to learn semantically rich representations without manual source annotations. During downstream adaptation, target-label-set-aware fine-tuning combines supervised learning from labeled target videos with zero-shot distillation over the actual target label set, helping preserve VLM-derived semantic guidance while adapting the pretrained encoder to the target task. Experiments on UCF101 and HMDB51 show that the proposed framework outperforms the compared semi-supervised video action recognition methods across all evaluated limited-label regimes. Moreover, the annotation-free pretraining stage learns transferable representations that provide an effective initialization for full-data fine-tuning, despite relying on a comparatively modest unlabeled pretraining pool.

Related papers

• Context-Aware Pseudo-Label Scoring for Zero-Shot Video Summarization [6.057968525653529]
  We propose a rubric-guided, pseudo-labeled, and prompt-driven zero-shot video summarization framework.<n>A small subset of human annotations is converted into high-confidence pseudo labels.<n>During inference, boundary scenes are scored independently based on their own descriptions.
  arXiv  Detail & Related papers  (2025-10-20T12:54:32Z)
• Bridging Weakly-Supervised Learning and VLM Distillation: Noisy Partial Label Learning for Efficient Downstream Adaptation [51.67328507400985]
  In noisy partial label learning (NPLL), each training sample is associated with a set of candidate labels annotated by multiple noisy annotators.<n>This paper focuses on learning from partial labels annotated by pre-trained vision-language models.<n>It proposes an innovative collaborative consistency regularization (Co-Reg) method.
  arXiv  Detail & Related papers  (2025-06-03T12:48:54Z)
• Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection [10.269746485037935]
  We propose a novel pseudo-label generation and self-training framework based on Text Prompt with Normality Guidance for WSVAD. Our method achieves state-of-the-art performance on two benchmark datasets, UCF-Crime and XD-Viole.
  arXiv  Detail & Related papers  (2024-04-12T15:18:25Z)
• CDFSL-V: Cross-Domain Few-Shot Learning for Videos [58.37446811360741]
  Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples. Existing methods in video action recognition rely on large labeled datasets from the same domain. We propose a novel cross-domain few-shot video action recognition method that leverages self-supervised learning and curriculum learning.
  arXiv  Detail & Related papers  (2023-09-07T19:44:27Z)
• Helping Hands: An Object-Aware Ego-Centric Video Recognition Model [60.350851196619296]
  We introduce an object-aware decoder for improving the performance of ego-centric representations on ego-centric videos. We show that the model can act as a drop-in replacement for an ego-awareness video model to improve performance through visual-text grounding.
  arXiv  Detail & Related papers  (2023-08-15T17:58:11Z)
• Simplifying Open-Set Video Domain Adaptation with Contrastive Learning [16.72734794723157]
  unsupervised video domain adaptation methods have been proposed to adapt a predictive model from a labelled dataset to an unlabelled dataset. We address a more realistic scenario, called open-set video domain adaptation (OUVDA), where the target dataset contains "unknown" semantic categories that are not shared with the source. We propose a video-oriented temporal contrastive loss that enables our method to better cluster the feature space by exploiting the freely available temporal information in video data.
  arXiv  Detail & Related papers  (2023-01-09T13:16:50Z)
• Domain Adaptive Video Segmentation via Temporal Pseudo Supervision [46.38660541271893]
  Video semantic segmentation can mitigate data labelling constraints by adapting from a labelled source domain toward an unlabelled target domain. We design temporal pseudo supervision (TPS), a simple and effective method that explores the idea of consistency training for representations effective from target videos. We show that TPS is simpler to implement, much more stable to train, and achieves superior video accuracy as compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-06T00:36:14Z)
• (Un)likelihood Training for Interpretable Embedding [30.499562324921648]
  Cross-modal representation learning has become a new normal for bridging the semantic gap between text and visual data. We propose two novel training objectives, likelihood and unlikelihood functions, to unroll semantics behind embeddings. With both training objectives, a new encoder-decoder network, which learns interpretable cross-modal representation, is proposed for ad-hoc video search.
  arXiv  Detail & Related papers  (2022-07-01T09:15:02Z)
• Prefix Conditioning Unifies Language and Label Supervision [84.11127588805138]
  We show that dataset biases negatively affect pre-training by reducing the generalizability of learned representations. In experiments, we show that this simple technique improves the performance in zero-shot image recognition accuracy and robustness to the image-level distribution shift.
  arXiv  Detail & Related papers  (2022-06-02T16:12:26Z)
• Align and Prompt: Video-and-Language Pre-training with Entity Prompts [111.23364631136339]
  Video-and-language pre-training has shown promising improvements on various downstream tasks. We propose Align and Prompt: an efficient and effective video-and-language pre-training framework with better cross-modal alignment. Our code and pre-trained models will be released.
  arXiv  Detail & Related papers  (2021-12-17T15:55:53Z)
• CUPID: Adaptive Curation of Pre-training Data for Video-and-Language Representation Learning [49.18591896085498]
  We propose CUPID to bridge the domain gap between source and target data. CUPID yields new state-of-the-art performance across multiple video-language and video tasks.
  arXiv  Detail & Related papers  (2021-04-01T06:42:16Z)
• Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts [89.06560404218028]
  We introduce a new method for pre-training video action recognition models using queried web videos. Instead of trying to filter out, we propose to convert the potential noises in these queried videos to useful supervision signals. We show that SPL outperforms several existing pre-training strategies using pseudo-labels.
  arXiv  Detail & Related papers  (2021-01-11T05:50:16Z)
• Learning Video Object Segmentation from Unlabeled Videos [158.18207922363783]
  We propose a new method for video object segmentation (VOS) that addresses object pattern learning from unlabeled videos. We introduce a unified unsupervised/weakly supervised learning framework, called MuG, that comprehensively captures properties of VOS at multiple granularities.
  arXiv  Detail & Related papers  (2020-03-10T22:12:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.