Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts

• URL: http://arxiv.org/abs/2407.15050v1
• Date: Sun, 21 Jul 2024 04:37:11 GMT
• Title: Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
• Authors: Yi Liu, Chengjun Cai, Xiaoli Zhang, Xingliang Yuan, Cong Wang,
• Abstract summary: Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of Large Language Models (LLMs) These advancements raise significant security and ethical concerns, particularly regarding the generation of harmful content. We introduce Arondight, a standardized red team framework tailored specifically for VLMs.
• Score: 25.661444231400772
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of Large Language Models (LLMs). Despite offering new possibilities for LLM applications, these advancements raise significant security and ethical concerns, particularly regarding the generation of harmful content. While LLMs have undergone extensive security evaluations with the aid of red teaming frameworks, VLMs currently lack a well-developed one. To fill this gap, we introduce Arondight, a standardized red team framework tailored specifically for VLMs. Arondight is dedicated to resolving issues related to the absence of visual modality and inadequate diversity encountered when transitioning existing red teaming methodologies from LLMs to VLMs. Our framework features an automated multi-modal jailbreak attack, wherein visual jailbreak prompts are produced by a red team VLM, and textual prompts are generated by a red team LLM guided by a reinforcement learning agent. To enhance the comprehensiveness of VLM security evaluation, we integrate entropy bonuses and novelty reward metrics. These elements incentivize the RL agent to guide the red team LLM in creating a wider array of diverse and previously unseen test cases. Our evaluation of ten cutting-edge VLMs exposes significant security vulnerabilities, particularly in generating toxic images and aligning multi-modal prompts. In particular, our Arondight achieves an average attack success rate of 84.5\% on GPT-4 in all fourteen prohibited scenarios defined by OpenAI in terms of generating toxic text. For a clearer comparison, we also categorize existing VLMs based on their safety levels and provide corresponding reinforcement recommendations. Our multimodal prompt dataset and red team code will be released after ethics committee approval. CONTENT WARNING: THIS PAPER CONTAINS HARMFUL MODEL RESPONSES.

Related papers

• IDEATOR: Jailbreaking VLMs Using VLMs [68.4760494411687]
  We propose a novel jailbreak method named IDEATOR, which autonomously generates malicious image-text pairs for black-box jailbreak attacks. IDEATOR employs a VLM to generate jailbreak texts while leveraging a state-of-the-art diffusion model to create corresponding jailbreak images. It successfully jailbreaks MiniGPT-4 with a 94% success rate and transfers seamlessly to LLaVA and InstructBLIP, achieving high success rates of 82% and 88%, respectively.
  arXiv  Detail & Related papers  (2024-10-29T07:15:56Z)
• RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent [24.487441771427434]
  We propose a multi-agent LLM system named RedAgent to generate context-aware jailbreak prompts. Our system can jailbreak most black-box LLMs in just five queries, improving the efficiency of existing red teaming methods by two times. We have reported all found issues and communicated with OpenAI and Meta for bug fixes.
  arXiv  Detail & Related papers  (2024-07-23T17:34:36Z)
• Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs) [17.670925982912312]
  Red-teaming is a technique for identifying vulnerabilities in large language models (LLM) This paper presents a detailed threat model and provides a systematization of knowledge (SoK) of red-teaming attacks on LLMs.
  arXiv  Detail & Related papers  (2024-07-20T17:05:04Z)
• A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends [78.3201480023907]
  Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across a wide range of multimodal understanding and reasoning tasks. The vulnerability of LVLMs is relatively underexplored, posing potential security risks in daily usage. In this paper, we provide a comprehensive review of the various forms of existing LVLM attacks.
  arXiv  Detail & Related papers  (2024-07-10T06:57:58Z)
• Automated Progressive Red Teaming [38.723546092060666]
  Manual red teaming is time-consuming, costly and lacks scalability. We propose Automated Progressive Red Teaming (APRT) as an effectively learnable framework. APRT leverages three core modules: an Intention Expanding LLM that generates diverse initial attack samples, an Intention Hiding LLM that crafts adversarial prompts, and an Evil Maker to manage prompt diversity and filter ineffective samples.
  arXiv  Detail & Related papers  (2024-07-04T12:14:27Z)
• Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks [41.213482317141356]
  Augmenting Large Language Models with image-understanding capabilities has resulted in a boom of high-performing Vision-Language models (VLMs) In this paper, we explore the impact of jailbreaking on three state-of-the-art VLMs, each using a distinct modeling approach.
  arXiv  Detail & Related papers  (2024-05-07T15:29:48Z)
• AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting [54.931241667414184]
  We propose textbfAdaptive textbfShield Prompting, which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks. Our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks.
  arXiv  Detail & Related papers  (2024-03-14T15:57:13Z)
• MART: Improving LLM Safety with Multi-round Automatic Red-Teaming [72.2127916030909]
  We propose a Multi-round Automatic Red-Teaming (MART) method, which incorporates both automatic adversarial prompt writing and safe response generation. On adversarial prompt benchmarks, the violation rate of an LLM with limited safety alignment reduces up to 84.7% after 4 rounds of MART. Notably, model helpfulness on non-adversarial prompts remains stable throughout iterations, indicating the target LLM maintains strong performance on instruction following.
  arXiv  Detail & Related papers  (2023-11-13T19:13:29Z)
• Attack Prompt Generation for Red Teaming and Defending Large Language Models [70.157691818224]
  Large language models (LLMs) are susceptible to red teaming attacks, which can induce LLMs to generate harmful content. We propose an integrated approach that combines manual and automatic methods to economically generate high-quality attack prompts.
  arXiv  Detail & Related papers  (2023-10-19T06:15:05Z)
• On Evaluating Adversarial Robustness of Large Vision-Language Models [64.66104342002882]
  We evaluate the robustness of large vision-language models (VLMs) in the most realistic and high-risk setting. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP. Black-box queries on these VLMs can further improve the effectiveness of targeted evasion.
  arXiv  Detail & Related papers  (2023-05-26T13:49:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.