PAL: Proxy-Guided Black-Box Attack on Large Language Models

• URL: http://arxiv.org/abs/2402.09674v1
• Date: Thu, 15 Feb 2024 02:54:49 GMT
• Title: PAL: Proxy-Guided Black-Box Attack on Large Language Models
• Authors: Chawin Sitawarin, Norman Mu, David Wagner, Alexandre Araujo
• Abstract summary: Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated capabilities to generate harmful content when manipulated. We introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting. Our attack achieves 84% attack success rate (ASR) on GPT-3.5-Turbo and 48% on Llama-2-7B, compared to 4% for the current state of the art.
• Score: 55.57987172146731
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated concerning capabilities to generate harmful content when manipulated. While techniques like safety fine-tuning aim to minimize harmful use, recent works have shown that LLMs remain vulnerable to attacks that elicit toxic responses. In this work, we introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting. In particular, it relies on a surrogate model to guide the optimization and a sophisticated loss designed for real-world LLM APIs. Our attack achieves 84% attack success rate (ASR) on GPT-3.5-Turbo and 48% on Llama-2-7B, compared to 4% for the current state of the art. We also propose GCG++, an improvement to the GCG attack that reaches 94% ASR on white-box Llama-2-7B, and the Random-Search Attack on LLMs (RAL), a strong but simple baseline for query-based attacks. We believe the techniques proposed in this work will enable more comprehensive safety testing of LLMs and, in the long term, the development of better security guardrails. The code can be found at https://github.com/chawins/pal.

Related papers

• LLMStinger: Jailbreaking LLMs using RL fine-tuned LLMs [13.36946005380889]
  We introduce LLMStinger, a novel approach that leverages Large Language Models (LLMs) to automatically generate adversarial suffixes for jailbreak attacks. Our method significantly outperforms existing red-teaming approaches, achieving a +57.2% improvement in Attack Success Rate (ASR) on LLaMA2-7B-chat and a +50.3% increase on Claude 2.
  arXiv  Detail & Related papers  (2024-11-13T18:44:30Z)
• Denial-of-Service Poisoning Attacks against Large Language Models [64.77355353440691]
  LLMs are vulnerable to denial-of-service (DoS) attacks, where spelling errors or non-semantic prompts trigger endless outputs without generating an [EOS] token. We propose poisoning-based DoS attacks for LLMs, demonstrating that injecting a single poisoned sample designed for DoS purposes can break the output length limit.
  arXiv  Detail & Related papers  (2024-10-14T17:39:31Z)
• MaPPing Your Model: Assessing the Impact of Adversarial Attacks on LLM-based Programming Assistants [14.947665219536708]
  We introduce the Malicious Programming Prompt (MaPP) attack, in which an attacker adds a small amount of text to a prompt for a programming task. We show that our prompt strategy can cause an LLM to add vulnerabilities while continuing to write otherwise correct code.
  arXiv  Detail & Related papers  (2024-07-12T22:30:35Z)
• QROA: A Black-Box Query-Response Optimization Attack on LLMs [2.7624021966289605]
  Large Language Models (LLMs) have surged in popularity in recent months, yet they possess capabilities for generating harmful content when manipulated. This study introduces the Query-Response Optimization Attack (QROA), an optimization-based strategy designed to exploit LLMs through a black-box, query-only interaction.
  arXiv  Detail & Related papers  (2024-06-04T07:27:36Z)
• Improved Generation of Adversarial Examples Against Safety-aligned LLMs [72.38072942860309]
  Adversarial prompts generated using gradient-based methods exhibit outstanding performance in performing automatic jailbreak attacks against safety-aligned LLMs. In this paper, we explore a new perspective on this problem, suggesting that it can be alleviated by leveraging innovations inspired in transfer-based attacks. We show that 87% of the query-specific adversarial suffixes generated by the developed combination can induce Llama-2-7B-Chat to produce the output that exactly matches the target string on AdvBench.
  arXiv  Detail & Related papers  (2024-05-28T06:10:12Z)
• Bypassing the Safety Training of Open-Source LLMs with Priming Attacks [3.8023902618391783]
  In this paper, we investigate the fragility of SOTA open-source LLMs under simple, optimization-free attacks. Our proposed attack improves the Attack Success Rate on Harmful Behaviors, as measured by Llama Guard, by up to $3.3times$ compared to baselines.
  arXiv  Detail & Related papers  (2023-12-19T16:47:12Z)
• Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization [98.18718484152595]
  We propose to integrate goal prioritization at both training and inference stages to counteract the intrinsic conflict between the goals of being helpful and ensuring safety. Our work thus contributes to the comprehension of jailbreaking attacks and defenses, and sheds light on the relationship between LLMs' capability and safety.
  arXiv  Detail & Related papers  (2023-11-15T16:42:29Z)
• Attack Prompt Generation for Red Teaming and Defending Large Language Models [70.157691818224]
  Large language models (LLMs) are susceptible to red teaming attacks, which can induce LLMs to generate harmful content. We propose an integrated approach that combines manual and automatic methods to economically generate high-quality attack prompts.
  arXiv  Detail & Related papers  (2023-10-19T06:15:05Z)
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
  We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs) Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
  arXiv  Detail & Related papers  (2023-10-05T17:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.