Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Models

• URL: http://arxiv.org/abs/2411.07559v1
• Date: Tue, 12 Nov 2024 05:24:02 GMT
• Title: Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Models
• Authors: Tiejin Chen, Kaishen Wang, Hua Wei,
• Abstract summary: We propose Zer0-Jack, a method that bypasses the need for white-box access by leveraging zeroth-order optimization. Zer0-Jack achieves a high attack success rate across various models, surpassing previous transfer-based methods. We show that Zer0-Jack can directly attack commercial MLLMs such as GPT-4o.
• Score: 2.740881223898167
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Jailbreaking methods, which induce Multi-modal Large Language Models (MLLMs) to output harmful responses, raise significant safety concerns. Among these methods, gradient-based approaches, which use gradients to generate malicious prompts, have been widely studied due to their high success rates in white-box settings, where full access to the model is available. However, these methods have notable limitations: they require white-box access, which is not always feasible, and involve high memory usage. To address scenarios where white-box access is unavailable, attackers often resort to transfer attacks. In transfer attacks, malicious inputs generated using white-box models are applied to black-box models, but this typically results in reduced attack performance. To overcome these challenges, we propose Zer0-Jack, a method that bypasses the need for white-box access by leveraging zeroth-order optimization. We propose patch coordinate descent to efficiently generate malicious image inputs to directly attack black-box MLLMs, which significantly reduces memory usage further. Through extensive experiments, Zer0-Jack achieves a high attack success rate across various models, surpassing previous transfer-based methods and performing comparably with existing white-box jailbreak techniques. Notably, Zer0-Jack achieves a 95\% attack success rate on MiniGPT-4 with the Harmful Behaviors Multi-modal Dataset on a black-box setting, demonstrating its effectiveness. Additionally, we show that Zer0-Jack can directly attack commercial MLLMs such as GPT-4o. Codes are provided in the supplement.

Related papers

• Prefill-Based Jailbreak: A Novel Approach of Bypassing LLM Safety Boundary [2.4329261266984346]
  Large Language Models (LLMs) are designed to generate helpful and safe content. adversarial attacks, commonly referred to as jailbreak, can bypass their safety protocols. We introduce a novel jailbreak attack method that leverages the prefilling feature of LLMs.
  arXiv  Detail & Related papers  (2025-04-28T07:38:43Z)
• Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation [71.92055093709924]
  We propose a novel method that "translates" garbled adversarial prompts into coherent and human-readable natural language adversarial prompts. It also offers a new approach to discovering effective designs for jailbreak prompts, advancing the understanding of jailbreak attacks. Our method achieves over 90% attack success rates against Llama-2-Chat models on AdvBench, despite their outstanding resistance to jailbreak attacks.
  arXiv  Detail & Related papers  (2024-10-15T06:31:04Z)
• RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking [30.67803190789498]
  We propose a new jailbreak approach, RED QUEEN ATTACK, that constructs a multi-turn scenario, concealing the malicious intent under the guise of preventing harm. Our experiments reveal that all LLMs are vulnerable to RED QUEEN ATTACK, reaching 87.62% attack success rate on GPT-4o and 75.4% on Llama3-70B. To prioritize safety, we introduce a straightforward mitigation strategy called RED QUEEN GUARD, which aligns LLMs to effectively counter adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-26T01:24:17Z)
• AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs [34.221522224051846]
  We propose an adaptive position pre-fill jailbreak attack approach for executing jailbreak attacks on Large Language Models (LLMs) Our method leverages the model's instruction-following capabilities to first output safe content, then exploits its narrative-shifting abilities to generate harmful content. Our method can improve the attack success rate by 47% on the widely recognized secure model (Llama2) compared to existing approaches.
  arXiv  Detail & Related papers  (2024-09-11T00:00:58Z)
• Query Efficient Cross-Dataset Transferable Black-Box Attack on Action Recognition [99.29804193431823]
  Black-box adversarial attacks present a realistic threat to action recognition systems. We propose a new attack on action recognition that addresses these shortcomings by generating perturbations. Our method achieves 8% and higher 12% deception rates compared to state-of-the-art query-based and transfer-based attacks.
  arXiv  Detail & Related papers  (2022-11-23T17:47:49Z)
• T-SEA: Transfer-based Self-Ensemble Attack on Object Detection [9.794192858806905]
  We propose a single-model transfer-based black-box attack on object detection, utilizing only one model to achieve a high-transferability adversarial attack on multiple black-box detectors. We analogize patch optimization with regular model optimization, proposing a series of self-ensemble approaches on the input data, the attacked model, and the adversarial patch.
  arXiv  Detail & Related papers  (2022-11-16T10:27:06Z)
• Boosting Black-Box Adversarial Attacks with Meta Learning [0.0]
  We propose a hybrid attack method which trains meta adversarial perturbations (MAPs) on surrogate models and performs black-box attacks by estimating gradients of the models. Our method can not only improve the attack success rates, but also reduces the number of queries compared to other methods.
  arXiv  Detail & Related papers  (2022-03-28T09:32:48Z)
• Universal Adversarial Attacks on Neural Networks for Power Allocation in a Massive MIMO System [60.46526086158021]
  We propose universal adversarial perturbation (UAP)-crafting methods as white-box and black-box attacks. We show that the adversarial success rate can achieve up to 60% and 40%, respectively. The proposed UAP-based attacks make a more practical and realistic approach as compared to classical white-box attacks.
  arXiv  Detail & Related papers  (2021-10-10T08:21:03Z)
• Meta Gradient Adversarial Attack [64.5070788261061]
  This paper proposes a novel architecture called Metaversa Gradient Adrial Attack (MGAA), which is plug-and-play and can be integrated with any existing gradient-based attack method. Specifically, we randomly sample multiple models from a model zoo to compose different tasks and iteratively simulate a white-box attack and a black-box attack in each task. By narrowing the gap between the gradient directions in white-box and black-box attacks, the transferability of adversarial examples on the black-box setting can be improved.
  arXiv  Detail & Related papers  (2021-08-09T17:44:19Z)
• Spanning Attack: Reinforce Black-box Attacks with Unlabeled Data [96.92837098305898]
  Black-box attacks aim to craft adversarial perturbations by querying input-output pairs of machine learning models. Black-box attacks often suffer from the issue of query inefficiency due to the high dimensionality of the input space. We propose a novel technique called the spanning attack, which constrains adversarial perturbations in a low-dimensional subspace via spanning an auxiliary unlabeled dataset.
  arXiv  Detail & Related papers  (2020-05-11T05:57:15Z)
• Diversity can be Transferred: Output Diversification for White- and Black-box Attacks [89.92353493977173]
  Adrial attacks often involve random perturbations of the inputs drawn from uniform or Gaussian distributions, e.g., to initialize optimization-based white-box attacks or generate update directions in black-box attacks. We propose Output Diversified Sampling (ODS), a novel sampling strategy that attempts to maximize diversity in the target model's outputs among the generated samples. ODS significantly improves the performance of existing white-box and black-box attacks. In particular, ODS reduces the number of queries needed for state-of-the-art black-box attacks on ImageNet by a factor of two.
  arXiv  Detail & Related papers  (2020-03-15T17:49:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.