Adversarial Backdoor Defense in CLIP

• URL: http://arxiv.org/abs/2409.15968v1
• Date: Tue, 24 Sep 2024 10:56:18 GMT
• Title: Adversarial Backdoor Defense in CLIP
• Authors: Junhao Kuang, Siyuan Liang, Jiawei Liang, Kuanrong Liu, Xiaochun Cao,
• Abstract summary: Multimodal contrastive pretraining, exemplified by models like CLIP, has been found to be vulnerable to backdoor attacks. We propose Adversarial Backdoor Defense, a novel data augmentation strategy that aligns features with meticulously crafted adversarial examples. Our experiments demonstrate that ABD provides robust defense against both traditional uni-modal and multimodal backdoor attacks targeting CLIP.
• Score: 47.6497532581449
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal contrastive pretraining, exemplified by models like CLIP, has been found to be vulnerable to backdoor attacks. While current backdoor defense methods primarily employ conventional data augmentation to create augmented samples aimed at feature alignment, these methods fail to capture the distinct features of backdoor samples, resulting in suboptimal defense performance. Observations reveal that adversarial examples and backdoor samples exhibit similarities in the feature space within the compromised models. Building on this insight, we propose Adversarial Backdoor Defense (ABD), a novel data augmentation strategy that aligns features with meticulously crafted adversarial examples. This approach effectively disrupts the backdoor association. Our experiments demonstrate that ABD provides robust defense against both traditional uni-modal and multimodal backdoor attacks targeting CLIP. Compared to the current state-of-the-art defense method, CleanCLIP, ABD reduces the attack success rate by 8.66% for BadNet, 10.52% for Blended, and 53.64% for BadCLIP, while maintaining a minimal average decrease of just 1.73% in clean accuracy.

Related papers

• Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats [52.94388672185062]
  We propose an efficient defense mechanism against backdoor threats using a concept known as machine unlearning. This entails strategically creating a small set of poisoned samples to aid the model's rapid unlearning of backdoor vulnerabilities. In the backdoor unlearning process, we present a novel token-based portion unlearning training regime.
  arXiv  Detail & Related papers  (2024-09-29T02:55:38Z)
• Towards Unified Robustness Against Both Backdoor and Adversarial Attacks [31.846262387360767]
  Deep Neural Networks (DNNs) are known to be vulnerable to both backdoor and adversarial attacks. This paper reveals that there is an intriguing connection between backdoor and adversarial attacks. A novel Progressive Unified Defense algorithm is proposed to defend against backdoor and adversarial attacks simultaneously.
  arXiv  Detail & Related papers  (2024-05-28T07:50:00Z)
• Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack [32.74007523929888]
  We re-investigate the characteristics of backdoored models after defense. We find that the original backdoors still exist in defense models derived from existing post-training defense strategies. We empirically show that these dormant backdoors can be easily re-activated during inference.
  arXiv  Detail & Related papers  (2024-05-25T08:57:30Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples [67.66153875643964]
  Backdoor attacks are serious security threats to machine learning models. In this paper, we explore the task of purifying a backdoored model using a small clean dataset. By establishing the connection between backdoor risk and adversarial risk, we derive a novel upper bound for backdoor risk.
  arXiv  Detail & Related papers  (2023-07-20T03:56:04Z)
• Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks [52.26631767748843]
  We propose ROCLIP, the first effective method for robust pre-training multimodal vision-language models against targeted data poisoning and backdoor attacks. ROCLIP effectively breaks the association between poisoned image-caption pairs by considering a relatively large and varying pool of random captions. Our experiments show that ROCLIP renders state-of-the-art targeted data poisoning and backdoor attacks ineffective during pre-training CLIP models.
  arXiv  Detail & Related papers  (2023-03-13T04:49:46Z)
• Backdoor Attack against NLP models with Robustness-Aware Perturbation defense [0.0]
  Backdoor attack intends to embed hidden backdoor into deep neural networks (DNNs) In our work, we break this defense by controlling the robustness gap between poisoned and clean samples using adversarial training step.
  arXiv  Detail & Related papers  (2022-04-08T10:08:07Z)
• On the Effectiveness of Adversarial Training against Backdoor Attacks [111.8963365326168]
  A backdoored model always predicts a target class in the presence of a predefined trigger pattern. In general, adversarial training is believed to defend against backdoor attacks. We propose a hybrid strategy which provides satisfactory robustness across different backdoor attacks.
  arXiv  Detail & Related papers  (2022-02-22T02:24:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.