Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models

• URL: http://arxiv.org/abs/2603.11149v2
• Date: Wed, 18 Mar 2026 04:23:56 GMT
• Title: Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models
• Abstract summary: Large language models remain vulnerable to jailbreak attacks, yet we still lack a systematic understanding of how jailbreak success scales with attacker effort across methods, model families, and harm types.<n>We initiate a scaling-law framework for jailbreaks by treating each attack as a compute-bounded optimization procedure and measuring progress on a shared FLOPs axis.<n>Our systematic evaluation spans four representative jailbreak paradigms, covering optimization-based attacks, self-refinement prompting, sampling-based selection, and genetic optimization.
• Score: 15.425738252512362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models remain vulnerable to jailbreak attacks, yet we still lack a systematic understanding of how jailbreak success scales with attacker effort across methods, model families, and harm types. We initiate a scaling-law framework for jailbreaks by treating each attack as a compute-bounded optimization procedure and measuring progress on a shared FLOPs axis. Our systematic evaluation spans four representative jailbreak paradigms, covering optimization-based attacks, self-refinement prompting, sampling-based selection, and genetic optimization, across multiple model families and scales on a diverse set of harmful goals. We investigate scaling laws that relate attacker budget to attack success score by fitting a simple saturating exponential function to FLOPs--success trajectories, and we derive comparable efficiency summaries from the fitted curves. Empirically, prompting-based paradigms tend to be the most compute-efficient compared to optimization-based methods. To explain this gap, we cast prompt-based updates into an optimization view and show via a same-state comparison that prompt-based attacks more effectively optimize in prompt space. We also show that attacks occupy distinct success--stealthiness operating points with prompting-based methods occupying the high-success, high-stealth region. Finally, we find that vulnerability is strongly goal-dependent: harms involving misinformation are typically easier to elicit than other non-misinformation harms.

Related papers

• Adversarial Reframing: A Framework for Targeted Generation in Language Models [4.231181719953953]
  Large Language Models (LLMs) are widely deployed in diverse real-world settings, yet remain vulnerable to jailbreaking, where prompt-based attacks safety filters.<n>We present THREAT (edversa bypass generation via Reframing) as a tool for proactively strengthening the safety of foundation models.
  arXiv  Detail & Related papers  (2026-05-20T19:31:07Z)
• Low Rank Adaptation for Adversarial Perturbation [50.24377354110586]
  Low-Rank Adaptation (LoRA) has significantly improved the training efficiency of Large Language Models (LLMs)<n>Since the generation of adversarial examples is an optimization process analogous to model training, this naturally raises the question: Do adversarial perturbations exhibit a similar low-rank structure?<n>In this paper, we provide both theoretical analysis and extensive empirical investigation across various attack methods, model architectures, and datasets to show that adversarial perturbations indeed possess a low-rank structure.
  arXiv  Detail & Related papers  (2026-04-30T06:38:28Z)
• TAO-Attack: Toward Advanced Optimization-Based Jailbreak Attacks for Large Language Models [15.495882533240833]
  Large language models (LLMs) have achieved remarkable success across diverse applications but remain vulnerable to jailbreak attacks.<n>We propose TAO-Attack, a new optimization-based jailbreak method.<n> TAO-Attack consistently outperforms state-of-the-art methods, achieving higher attack success rates and even reaching 100% in certain scenarios.
  arXiv  Detail & Related papers  (2026-03-03T15:25:53Z)
• Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent [1.1187085721899017]
  Large language models (LLMs) are increasingly deployed in critical applications.<n>LLMs remain vulnerable to jailbreak attacks enabled by crafted adversarial triggers appended to user prompts.<n>We propose an intrinsic optimization method which directly optimize relaxed one-hot encodings of the adversarial suffix tokens.
  arXiv  Detail & Related papers  (2025-08-20T17:03:32Z)
• Sampling-aware Adversarial Attacks Against Large Language Models [52.30089653615172]
  Existing adversarial attacks typically target harmful responses in single-point greedy generations.<n>We show that for the goal of eliciting harmful responses, repeated sampling of model outputs during the attack prompt optimization.<n>We show that integrating sampling into existing attacks boosts success rates by up to 37% and improves efficiency by up to two orders of magnitude.
  arXiv  Detail & Related papers  (2025-07-06T16:13:33Z)
• T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks [67.91652526657599]
  We formalize the T2V jailbreak attack as a discrete optimization problem and propose a joint objective-based optimization framework, called T2V-OptJail.<n>We conduct large-scale experiments on several T2V models, covering both open-source models and real commercial closed-source models.<n>The proposed method improves 11.4% and 10.0% over the existing state-of-the-art method in terms of attack success rate.
  arXiv  Detail & Related papers  (2025-05-10T16:04:52Z)
• An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks [87.64278063236847]
  In this work, we propose a unified threat model for the principled comparison of jailbreak attacks.<n>Our threat model checks if a given jailbreak is likely to occur in the distribution of text.<n>We adapt popular attacks to this threat model, and, for the first time, benchmark these attacks on equal footing with it.
  arXiv  Detail & Related papers  (2024-10-21T17:27:01Z)
• AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lens [83.08119913279488]
  We present a systematic analysis of the dependency relationships in jailbreak attack and defense techniques. We propose three comprehensive, automated, and logical frameworks. We show that the proposed ensemble jailbreak attack and defense framework significantly outperforms existing research.
  arXiv  Detail & Related papers  (2024-06-06T07:24:41Z)
• Preference Poisoning Attacks on Reward Model Learning [47.00395978031771]
  We investigate the nature and extent of a vulnerability in learning reward models from pairwise comparisons. We propose two classes of algorithmic approaches for these attacks: a gradient-based framework, and several variants of rank-by-distance methods. We find that the best attacks are often highly successful, achieving in the most extreme case 100% success rate with only 0.3% of the data poisoned.
  arXiv  Detail & Related papers  (2024-02-02T21:45:24Z)
• Versatile Weight Attack via Flipping Limited Bits [68.45224286690932]
  We study a novel attack paradigm, which modifies model parameters in the deployment stage. Considering the effectiveness and stealthiness goals, we provide a general formulation to perform the bit-flip based weight attack. We present two cases of the general formulation with different malicious purposes, i.e., single sample attack (SSA) and triggered samples attack (TSA)
  arXiv  Detail & Related papers  (2022-07-25T03:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.