Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

• URL: http://arxiv.org/abs/2603.11331v1
• Date: Wed, 11 Mar 2026 21:48:03 GMT
• Title: Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
• Abstract summary: Adrial attacks can reliably steer safety-aligned large language models toward unsafe behavior.<n>We propose a theoretical generative model of proxy language in terms of a spin-glass system operating in a replica-symmetry-breaking regime.<n>Within this framework, we analyze prompt injection-based jailbreaking.
• Score: 30.86966284669791
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial attacks can reliably steer safety-aligned large language models toward unsafe behavior. Empirically, we find that adversarial prompt-injection attacks can amplify attack success rate from the slow polynomial growth observed without injection to exponential growth with the number of inference-time samples. To explain this phenomenon, we propose a theoretical generative model of proxy language in terms of a spin-glass system operating in a replica-symmetry-breaking regime, where generations are drawn from the associated Gibbs measure and a subset of low-energy, size-biased clusters is designated unsafe. Within this framework, we analyze prompt injection-based jailbreaking. Short injected prompts correspond to a weak magnetic field aligned towards unsafe cluster centers and yield a power-law scaling of attack success rate with the number of inference-time samples, while long injected prompts, i.e., strong magnetic field, yield exponential scaling. We derive these behaviors analytically and confirm them empirically on large language models. This transition between two regimes is due to the appearance of an ordered phase in the spin chain under a strong magnetic field, which suggests that the injected jailbreak prompt enhances adversarial order in the language model.

Related papers

• What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics [4.336452316521159]
  Jailbreak attacks reveal a persistent weakness in aligned Large Language Models.<n>We investigate how harmful intent is encoded within the model's internal representations.<n>We find that jailbreak behavior is reflected in structured intermediate uncertainty dynamics.
  arXiv  Detail & Related papers  (2026-06-23T21:14:53Z)
• Furina: Fragmented Uncertainty-Driven Refusal Instability Attack [9.678677899786718]
  Safety alignment in large language models (LLMs) and multimodal large language models (MLLMs) is commonly assumed to operate as a near-binary threshold mechanism.<n>We challenge this assumption by revealing that safety behavior is governed by an instability region where small perturbations induce refusal decisions rather than deterministic outcomes.<n>We develop a multi-metric diagnostic framework combining external and internal signals to characterize this instability.<n>We introduce Furina, a jailbreak attack that deliberately induces this signature through fragmented, scene-anchored prompts without model-specific optimization.
  arXiv  Detail & Related papers  (2026-05-24T11:35:12Z)
• The Laminar Flow Hypothesis: Detecting Jailbreaks via Semantic Turbulence in Large Language Models [0.0]
  Laminar Flow Hypothesis: benign inputs induce smooth, gradual transitions in an LLM's high-dimensional latent space.<n> adversarial prompts trigger chaotic, high-variance trajectories - termed Semantic Turbulence.<n>Tests show that Semantic Turbulence serves not only as a lightweight, real-time jailbreak detector but also as a non-invasive diagnostic tool.
  arXiv  Detail & Related papers  (2025-12-14T18:10:29Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• Does More Inference-Time Compute Really Help Robustness? [50.47666612618054]
  We show that small-scale, open-source models can benefit from inference-time scaling.<n>We identify an important security risk, intuitively motivated and empirically verified as an inverse scaling law.<n>We urge practitioners to carefully weigh these subtle trade-offs before applying inference-time scaling in security-sensitive, real-world applications.
  arXiv  Detail & Related papers  (2025-07-21T18:08:38Z)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs [83.11815479874447]
  We propose a novel jailbreak attack framework, inspired by cognitive decomposition and biases in human cognition.<n>We employ cognitive decomposition to reduce the complexity of malicious prompts and relevance bias to reorganize prompts.<n>We also introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm.
  arXiv  Detail & Related papers  (2025-05-03T05:28:11Z)
• Feature-Aware Malicious Output Detection and Mitigation [8.378272216429954]
  We propose a feature-aware method for harmful response rejection (FMM)<n>FMM detects the presence of malicious features within the model's feature space and adaptively adjusts the model's rejection mechanism.<n> Experimental results demonstrate the effectiveness of our approach across multiple language models and diverse attack techniques.
  arXiv  Detail & Related papers  (2025-04-12T12:12:51Z)
• Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial States [0.0]
  Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks.<n>Yet they remain vulnerable to adversarial manipulations such as jailbreaking via prompt injection attacks.<n>We investigated the underlying latent subspaces of safe and jailbroken states by extracting hidden activations from a LLM.
  arXiv  Detail & Related papers  (2025-03-12T04:59:22Z)
• Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions [50.40122190627256]
  We introduce POATE, a novel jailbreak technique that harnesses contrastive reasoning to provoke unethical responses.<n>PoATE crafts semantically opposing intents and integrates them with adversarial templates, steering models toward harmful outputs with remarkable subtlety.<n>To counter this, we propose Intent-Aware CoT and Reverse Thinking CoT, which decompose queries to detect malicious intent and reason in reverse to evaluate and reject harmful responses.
  arXiv  Detail & Related papers  (2025-01-03T15:40:03Z)
• An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks [87.64278063236847]
  In this work, we propose a unified threat model for the principled comparison of jailbreak attacks.<n>Our threat model checks if a given jailbreak is likely to occur in the distribution of text.<n>We adapt popular attacks to this threat model, and, for the first time, benchmark these attacks on equal footing with it.
  arXiv  Detail & Related papers  (2024-10-21T17:27:01Z)
• Navigating the OverKill in Large Language Models [84.62340510027042]
  We investigate the factors for overkill by exploring how models handle and determine the safety of queries. Our findings reveal the presence of shortcuts within models, leading to an over-attention of harmful words like 'kill' and prompts emphasizing safety will exacerbate overkill. We introduce Self-Contrastive Decoding (Self-CD), a training-free and model-agnostic strategy, to alleviate this phenomenon.
  arXiv  Detail & Related papers  (2024-01-31T07:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.