Tastle: Distract Large Language Models for Automatic Jailbreak Attack
- URL: http://arxiv.org/abs/2403.08424v1
- Date: Wed, 13 Mar 2024 11:16:43 GMT
- Title: Tastle: Distract Large Language Models for Automatic Jailbreak Attack
- Authors: Zeguan Xiao, Yan Yang, Guanhua Chen, Yun Chen
- Abstract summary: We propose a black-box jailbreak framework for automated red teaming of large language models (LLMs)
Our framework is superior in terms of effectiveness, scalability and transferability.
We also evaluate the effectiveness of existing jailbreak defense methods against our attack.
- Score: 9.137714258654842
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large language models (LLMs) have achieved significant advances in recent
days. Extensive efforts have been made before the public release of LLMs to
align their behaviors with human values. The primary goal of alignment is to
ensure their helpfulness, honesty and harmlessness. However, even meticulously
aligned LLMs remain vulnerable to malicious manipulations such as jailbreaking,
leading to unintended behaviors. The jailbreak is to intentionally develop a
malicious prompt that escapes from the LLM security restrictions to produce
uncensored detrimental contents. Previous works explore different jailbreak
methods for red teaming LLMs, yet they encounter challenges regarding to
effectiveness and scalability. In this work, we propose Tastle, a novel
black-box jailbreak framework for automated red teaming of LLMs. We designed
malicious content concealing and memory reframing with an iterative
optimization algorithm to jailbreak LLMs, motivated by the research about the
distractibility and over-confidence phenomenon of LLMs. Extensive experiments
of jailbreaking both open-source and proprietary LLMs demonstrate the
superiority of our framework in terms of effectiveness, scalability and
transferability. We also evaluate the effectiveness of existing jailbreak
defense methods against our attack and highlight the crucial need to develop
more effective and practical defense strategies.
Related papers
- Virtual Context: Enhancing Jailbreak Attacks with Special Token Injection [54.05862550647966]
This paper introduces Virtual Context, which leverages special tokens, previously overlooked in LLM security, to improve jailbreak attacks.
Comprehensive evaluations show that Virtual Context-assisted jailbreak attacks can improve the success rates of four widely used jailbreak methods by approximately 40%.
arXiv Detail & Related papers (2024-06-28T11:35:54Z) - How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States [65.45603614354329]
Large language models (LLMs) rely on safety alignment to avoid responding to malicious user inputs.
Jailbreak can circumvent safety guardrails, resulting in LLMs generating harmful content.
We employ weak classifiers to explain LLM safety through the intermediate hidden states.
arXiv Detail & Related papers (2024-06-09T05:04:37Z) - Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks [59.46556573924901]
This paper introduces Defensive Prompt Patch (DPP), a novel prompt-based defense mechanism for large language models (LLMs)
Unlike previous approaches, DPP is designed to achieve a minimal Attack Success Rate (ASR) while preserving the high utility of LLMs.
Empirical results conducted on LLAMA-2-7B-Chat and Mistral-7B-Instruct-v0.2 models demonstrate the robustness and adaptability of DPP.
arXiv Detail & Related papers (2024-05-30T14:40:35Z) - Efficient LLM-Jailbreaking by Introducing Visual Modality [28.925716670778076]
This paper focuses on jailbreaking attacks against large language models (LLMs)
Our approach begins by constructing a multimodal large language model (MLLM) through the incorporation of a visual module into the target LLM.
We convert the embJS into text space to facilitate the jailbreaking of the target LLM.
arXiv Detail & Related papers (2024-05-30T12:50:32Z) - Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing [14.094372002702476]
Large language models (LLMs) are increasingly being adopted in a wide range of real-world applications.
Recent studies have shown that LLMs are vulnerable to deliberately crafted adversarial prompts.
We propose a novel defense method termed textbfLayer-specific textbfEditing (LED) to enhance the resilience of LLMs against jailbreak attacks.
arXiv Detail & Related papers (2024-05-28T13:26:12Z) - Open the Pandora's Box of LLMs: Jailbreaking LLMs through Representation
Engineering [44.10397472780012]
We propose a novel jailbreaking approach, named Jailbreaking LLMs through Representation Engineering (JRE)
Our method requires only a small number of query pairs to extract safety patterns'' that can be used to circumvent the target model's defenses.
Building upon these findings, we also introduce a novel defense framework inspired by JRE principles, which demonstrates notable effectiveness.
arXiv Detail & Related papers (2024-01-12T00:50:04Z) - A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily [51.63085197162279]
Large Language Models (LLMs) are designed to provide useful and safe responses.
adversarial prompts known as 'jailbreaks' can circumvent safeguards.
We propose ReNeLLM, an automatic framework that leverages LLMs themselves to generate effective jailbreak prompts.
arXiv Detail & Related papers (2023-11-14T16:02:16Z) - Jailbreaking Black Box Large Language Models in Twenty Queries [97.29563503097995]
Large language models (LLMs) are vulnerable to adversarial jailbreaks.
We propose an algorithm that generates semantic jailbreaks with only black-box access to an LLM.
arXiv Detail & Related papers (2023-10-12T15:38:28Z) - SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs)
Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
arXiv Detail & Related papers (2023-10-05T17:01:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.