Prompt Backdoors in Visual Prompt Learning

• URL: http://arxiv.org/abs/2310.07632v1
• Date: Wed, 11 Oct 2023 16:25:45 GMT
• Title: Prompt Backdoors in Visual Prompt Learning
• Abstract summary: We propose BadVisualPrompt, a simple yet effective backdoor attack against VPL. We identify and address a new technical challenge related to interactions between the backdoor trigger and visual prompt.
• Score: 35.90535387625355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fine-tuning large pre-trained computer vision models is infeasible for resource-limited users. Visual prompt learning (VPL) has thus emerged to provide an efficient and flexible alternative to model fine-tuning through Visual Prompt as a Service (VPPTaaS). Specifically, the VPPTaaS provider optimizes a visual prompt given downstream data, and downstream users can use this prompt together with the large pre-trained model for prediction. However, this new learning paradigm may also pose security risks when the VPPTaaS provider instead provides a malicious visual prompt. In this paper, we take the first step to explore such risks through the lens of backdoor attacks. Specifically, we propose BadVisualPrompt, a simple yet effective backdoor attack against VPL. For example, poisoning $5\%$ CIFAR10 training data leads to above $99\%$ attack success rates with only negligible model accuracy drop by $1.5\%$. In particular, we identify and then address a new technical challenge related to interactions between the backdoor trigger and visual prompt, which does not exist in conventional, model-level backdoors. Moreover, we provide in-depth analyses of seven backdoor defenses from model, prompt, and input levels. Overall, all these defenses are either ineffective or impractical to mitigate our BadVisualPrompt, implying the critical vulnerability of VPL.

Related papers

• Backdoor Vectors: a Task Arithmetic View on Backdoor Attacks and Defenses [2.82789527790216]
  We propose a simple framework for understanding backdoor attacks by treating the attack itself as a task vector.<n>We propose a novel method that enhances backdoor resilience through merging dubbed $Sparse Backdoor Vector (SBV)$ that combines multiple attacks into a single one.<n>Our results show that SBVs surpass prior attacks and is the first method to leverage merging to improve backdoor effectiveness.
  arXiv  Detail & Related papers  (2025-10-09T09:54:05Z)
• Neural Antidote: Class-Wise Prompt Tuning for Purifying Backdoors in Pre-trained Vision-Language Models [42.81731204702258]
  Class-wise Backdoor Prompt Tuning (CBPT) is an efficient and effective method that operates on the text prompts to indirectly purify poisoned Vision-Language Models (VLMs) CBPT significantly mitigates backdoor threats while preserving model utility, e.g. an average Clean Accuracy (CA) of 58.86% and an Attack Success Rate (ASR) of 0.39% across seven mainstream backdoor attacks.
  arXiv  Detail & Related papers  (2025-02-26T16:25:15Z)
• Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning [19.638259197558625]
  Multimodal contrastive learning models (e.g., CLIP) can learn high-quality representations from large-scale image-text datasets.<n>They exhibit significant vulnerabilities to backdoor attacks, raising serious safety concerns.<n>We propose Repulsive Visual Prompt Tuning (RVPT), a novel defense approach that employs deep visual prompt tuning with a specially designed feature-repelling loss.
  arXiv  Detail & Related papers  (2024-12-29T08:09:20Z)
• An Effective and Resilient Backdoor Attack Framework against Deep Neural Networks and Vision Transformers [22.77836113915616]
  We propose a novel attention-based mask generation methodology that searches for the optimal trigger shape and location. We also introduce a Quality-of-Experience term into the loss function and carefully adjust the transparency value of the trigger. Our proposed backdoor attack framework also showcases robustness against state-of-the-art backdoor defenses.
  arXiv  Detail & Related papers  (2024-12-09T02:03:27Z)
• Expose Before You Defend: Unifying and Enhancing Backdoor Defenses via Exposed Models [68.40324627475499]
  We introduce a novel two-step defense framework named Expose Before You Defend. EBYD unifies existing backdoor defense methods into a comprehensive defense system with enhanced performance. We conduct extensive experiments on 10 image attacks and 6 text attacks across 2 vision datasets and 4 language datasets.
  arXiv  Detail & Related papers  (2024-10-25T09:36:04Z)
• Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transformers [51.0477382050976]
  An extra prompt token, called the switch token in this work, can turn the backdoor mode on, converting a benign model into a backdoored one. To attack a pre-trained model, our proposed attack, named SWARM, learns a trigger and prompt tokens including a switch token. Experiments on diverse visual recognition tasks confirm the success of our switchable backdoor attack, achieving 95%+ attack success rate.
  arXiv  Detail & Related papers  (2024-05-17T08:19:48Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• BadPrompt: Backdoor Attacks on Continuous Prompts [16.780478840918995]
  We conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks. We propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts. We evaluate the performance of BadPrompt on five datasets and two continuous prompt models.
  arXiv  Detail & Related papers  (2022-11-27T04:23:18Z)
• Few-Shot Backdoor Attacks on Visual Object Tracking [80.13936562708426]
  Visual object tracking (VOT) has been widely adopted in mission-critical applications, such as autonomous driving and intelligent surveillance systems. We show that an adversary can easily implant hidden backdoors into VOT models by tempering with the training process. We show that our attack is resistant to potential defenses, highlighting the vulnerability of VOT models to potential backdoor attacks.
  arXiv  Detail & Related papers  (2022-01-31T12:38:58Z)
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks [46.99548490594115]
  A backdoor attack installs a backdoor into the victim model by injecting a backdoor pattern into a small proportion of the training data. We propose reflection backdoor (Refool) to plant reflections as backdoor into a victim model. We demonstrate on 3 computer vision tasks and 5 datasets that, Refool can attack state-of-the-art DNNs with high success rate.
  arXiv  Detail & Related papers  (2020-07-05T13:56:48Z)
• Blind Backdoors in Deep Learning Models [22.844973592524966]
  We investigate a new method for injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code. We use it to demonstrate new classes of backdoors strictly more powerful than those in the prior literature. Our attack is blind: the attacker cannot modify the training data, nor observe the execution of his code, nor access the resulting model.
  arXiv  Detail & Related papers  (2020-05-08T02:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.