EV-CLIP: Efficient Visual Prompt Adaptation for CLIP in Few-shot Action Recognition under Visual Challenges

• URL: http://arxiv.org/abs/2604.22595v1
• Date: Fri, 24 Apr 2026 14:23:24 GMT
• Title: EV-CLIP: Efficient Visual Prompt Adaptation for CLIP in Few-shot Action Recognition under Visual Challenges
• Abstract summary: Efficient Visual Prompting for CLIP (EV-CLIP) is an adaptation framework designed for few-shot video action recognition across diverse scenes and viewpoints.<n>EV-CLIP introduces two visual prompts: mask prompts, which guide the model's attention to action-relevant regions by reweighting pixels, and context prompts, which perform lightweight temporal modeling.<n> Experimental results demonstrate that EV-CLIP outperforms existing parameter-efficient methods in overall performance.
• Score: 1.9662978733004597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: CLIP has demonstrated strong generalization in visual domains through natural language supervision, even for video action recognition. However, most existing approaches that adapt CLIP for action recognition have primarily focused on temporal modeling, often overlooking spatial perception. In real-world scenarios, visual challenges such as low-light environments or egocentric viewpoints can severely impair spatial understanding, an essential precursor for effective temporal reasoning. To address this limitation, we propose Efficient Visual Prompting for CLIP (EV-CLIP), an efficient adaptation framework designed for few-shot video action recognition across diverse scenes and viewpoints. EV-CLIP introduces two visual prompts: mask prompts, which guide the model's attention to action-relevant regions by reweighting pixels, and context prompts, which perform lightweight temporal modeling by compressing frame-wise features into a compact representation. For a comprehensive evaluation, we curate five benchmark datasets and analyze domain shifts to quantify the influence of diverse visual and semantic factors on action recognition. Experimental results demonstrate that EV-CLIP outperforms existing parameter-efficient methods in overall performance. Moreover, its efficiency remains independent of the backbone scale, making it well-suited for deployment in real-world, resource-constrained scenarios. The code is available at https://github.com/AI-CV-Lab/EV-CLIP.

Related papers

• Fine-grained CLIP fine-tuning with self-annotated region alignment [57.28808560528696]
  Contrastive Language-Image Pre-training has been shown to have limitations in its fine-grained dense feature representation.<n>We propose SFF-CLIP (Self-annotated Fine-grained Fine-tuning for CLIP) to boost the fine-grained representation ability in the CLIP fine-tuning.
  arXiv  Detail & Related papers  (2026-07-15T10:06:39Z)
• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition [54.55914886780534]
  Event stream-based Visual Place Recognition (VPR) is an emerging research direction that offers a compelling solution to the instability of conventional visible-light cameras under challenging conditions such as low illumination, overexposure, and high-speed motion.<n>We introduce EPRBench, a high-quality benchmark specifically designed for event stream-based VPR.<n>EPRBench comprises 10K event sequences and 65K event frames, collected using both handheld and vehicle-mounted setups to comprehensively capture real-world challenges across diverse viewpoints, weather conditions, and lighting scenarios.
  arXiv  Detail & Related papers  (2026-02-13T13:25:05Z)
• Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception [71.26728044621458]
  DeCLIP is a novel framework that enhances CLIP by decoupling the self-attention module to obtain content'' and context'' features respectively.<n>It consistently achieves state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.
  arXiv  Detail & Related papers  (2025-08-15T06:43:51Z)
• Helping CLIP See Both the Forest and the Trees: A Decomposition and Description Approach [43.419607730361996]
  Vision-Language Models (VLMs) like CLIP achieve cross-modal alignment through contrastive learning.<n>Traditional prompt engineering relies on coarse-grained category labels, neglecting fine-grained local semantics.<n>We propose a plug-and-play solution that enables CLIP to process localized visual descriptors.
  arXiv  Detail & Related papers  (2025-07-04T10:24:26Z)
• Top-Down Compression: Revisit Efficient Vision Token Projection for Visual Instruction Tuning [70.57180215148125]
  Visual instruction tuning aims to enable large language models to comprehend the visual world.<n>Existing methods often grapple with the intractable trade-off between accuracy and efficiency.<n>We present LLaVA-Meteor, a novel approach that strategically compresses visual tokens without compromising core information.
  arXiv  Detail & Related papers  (2025-05-17T10:22:29Z)
• DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception [21.87721909270275]
  DeCLIP is a novel framework that enhances CLIP with content'' and context'' features respectively.<n>It significantly outperforms existing methods across multiple open-vocabulary dense prediction tasks.
  arXiv  Detail & Related papers  (2025-05-07T13:46:34Z)
• PE-CLIP: A Parameter-Efficient Fine-Tuning of Vision Language Models for Dynamic Facial Expression Recognition [7.966499123076283]
  Vision-Language Models (VLMs) like CLIP offer promising solutions for Dynamic Facial Expression Recognition (DFER)<n>We propose PE-CLIP, a parameter-efficient fine-tuning framework that adapts CLIP for DFER while significantly reducing trainable parameters.<n>By balancing efficiency and accuracy, PE-CLIP sets a new benchmark in resource-efficient DFER.
  arXiv  Detail & Related papers  (2025-03-21T08:45:50Z)
• VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection [58.47940430618352]
  We propose VadCLIP, a new paradigm for weakly supervised video anomaly detection (WSVAD) VadCLIP makes full use of fine-grained associations between vision and language on the strength of CLIP. We conduct extensive experiments on two commonly-used benchmarks, demonstrating that VadCLIP achieves the best performance on both coarse-grained and fine-grained WSVAD.
  arXiv  Detail & Related papers  (2023-08-22T14:58:36Z)
• MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognition [41.78245303513613]
  We introduce MA-FSAR, a framework that employs the Fine-Tuning (PEFT) technique to enhance the CLIP visual encoder in terms of action-related temporal and semantic representations. In addition to these token-level designs, we propose a prototype-level text-guided construction module to further enrich the temporal and semantic characteristics of video prototypes.
  arXiv  Detail & Related papers  (2023-08-03T04:17:25Z)
• Non-Contrastive Learning Meets Language-Image Pre-Training [145.6671909437841]
  We study the validity of non-contrastive language-image pre-training (nCLIP) We introduce xCLIP, a multi-tasking framework combining CLIP and nCLIP, and show that nCLIP aids CLIP in enhancing feature semantics.
  arXiv  Detail & Related papers  (2022-10-17T17:57:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.