Image Hijacks: Adversarial Images can Control Generative Models at Runtime

• URL: http://arxiv.org/abs/2309.00236v4
• Date: Tue, 17 Sep 2024 19:56:09 GMT
• Title: Image Hijacks: Adversarial Images can Control Generative Models at Runtime
• Authors: Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons,
• Abstract summary: We discover image hijacks, adversarial images that control the behaviour of vision-language models at inference time. We derive the Prompt Matching method, allowing us to train hijacks matching the behaviour of an arbitrary user-defined text prompt. We use Behaviour Matching to craft hijacks for four types of attack, forcing VLMs to generate outputs of the adversary's choice, leak information from their context window, override their safety training, and believe false statements.
• Score: 8.603201325413192
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Are foundation models secure against malicious actors? In this work, we focus on the image input to a vision-language model (VLM). We discover image hijacks, adversarial images that control the behaviour of VLMs at inference time, and introduce the general Behaviour Matching algorithm for training image hijacks. From this, we derive the Prompt Matching method, allowing us to train hijacks matching the behaviour of an arbitrary user-defined text prompt (e.g. 'the Eiffel Tower is now located in Rome') using a generic, off-the-shelf dataset unrelated to our choice of prompt. We use Behaviour Matching to craft hijacks for four types of attack, forcing VLMs to generate outputs of the adversary's choice, leak information from their context window, override their safety training, and believe false statements. We study these attacks against LLaVA, a state-of-the-art VLM based on CLIP and LLaMA-2, and find that all attack types achieve a success rate of over 80%. Moreover, our attacks are automated and require only small image perturbations.

Related papers

• AnyAttack: Towards Large-scale Self-supervised Generation of Targeted Adversarial Examples for Vision-Language Models [41.044385916368455]
  Vision-Language Models (VLMs) are vulnerable to image-based adversarial attacks. We propose AnyAttack, a self-supervised framework that generates targeted adversarial images for VLMs without label supervision.
  arXiv  Detail & Related papers  (2024-10-07T09:45:18Z)
• Adversarial Attacks on Multimodal Agents [73.97379283655127]
  Vision-enabled language models (VLMs) are now used to build autonomous multimodal agents capable of taking actions in real environments. We show that multimodal agents raise new safety risks, even though attacking agents is more challenging than prior attacks due to limited access to and knowledge about the environment.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)
• White-box Multimodal Jailbreaks Against Large Vision-Language Models [61.97578116584653]
  We propose a more comprehensive strategy that jointly attacks both text and image modalities to exploit a broader spectrum of vulnerability within Large Vision-Language Models. Our attack method begins by optimizing an adversarial image prefix from random noise to generate diverse harmful responses in the absence of text input. An adversarial text suffix is integrated and co-optimized with the adversarial image prefix to maximize the probability of eliciting affirmative responses to various harmful instructions.
  arXiv  Detail & Related papers  (2024-05-28T07:13:30Z)
• VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models [58.21452697997078]
  We propose a novel VQAttack model, which can generate both image and text perturbations with the designed modules. Experimental results on two VQA datasets with five validated models demonstrate the effectiveness of the proposed VQAttack.
  arXiv  Detail & Related papers  (2024-02-16T21:17:42Z)
• Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks [62.34019142949628]
  Typographic Attacks, which involve pasting misleading text onto an image, were noted to harm the performance of Vision-Language Models like CLIP. We introduce two novel and more effective textitSelf-Generated attacks which prompt the LVLM to generate an attack against itself. Using our benchmark, we uncover that Self-Generated attacks pose a significant threat, reducing LVLM(s) classification performance by up to 33%.
  arXiv  Detail & Related papers  (2024-02-01T14:41:20Z)
• InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models [13.21813503235793]
  Large vision-language models (LVLMs) have demonstrated their incredible capability in image understanding and response generation. In this paper, we formulate a novel and practical targeted attack scenario that the adversary can only know the vision encoder of the victim LVLM. We propose an instruction-tuned targeted attack (dubbed textscInstructTA) to deliver the targeted adversarial attack on LVLMs with high transferability.
  arXiv  Detail & Related papers  (2023-12-04T13:40:05Z)
• Universal and Transferable Adversarial Attacks on Aligned Language Models [118.41733208825278]
  We propose a simple and effective attack method that causes aligned language models to generate objectionable behaviors. Surprisingly, we find that the adversarial prompts generated by our approach are quite transferable.
  arXiv  Detail & Related papers  (2023-07-27T17:49:12Z)
• Fooling Contrastive Language-Image Pre-trained Models with CLIPMasterPrints [15.643898659673036]
  We show that despite their versatility, CLIP models are vulnerable to what we refer to as fooling master images. Fooling master images are capable of maximizing the confidence score of a CLIP model for a significant number of widely varying prompts. We demonstrate how fooling master images for CLIPMasterPrints can be mined using gradient descent, projected descent, or blackbox optimization.
  arXiv  Detail & Related papers  (2023-07-07T18:54:11Z)
• Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attacks [154.31827097264264]
  Adversarial training is a popular defense strategy against attack threat models with bounded Lp norms. We propose Dual Manifold Adversarial Training (DMAT) where adversarial perturbations in both latent and image spaces are used in robustifying the model. Our DMAT improves performance on normal images, and achieves comparable robustness to the standard adversarial training against Lp attacks.
  arXiv  Detail & Related papers  (2020-09-05T06:00:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.