What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics

• URL: http://arxiv.org/abs/2606.25182v1
• Date: Tue, 23 Jun 2026 21:14:53 GMT
• Title: What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics
• Abstract summary: Jailbreak attacks reveal a persistent weakness in aligned Large Language Models.<n>We investigate how harmful intent is encoded within the model's internal representations.<n>We find that jailbreak behavior is reflected in structured intermediate uncertainty dynamics.
• Score: 4.336452316521159
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced.

Related papers

• Defending against Model Extraction for GNNs with Model Reprogramming [57.82196268649236]
  Graph Neural Networks (GNNs) serve as the backbone for high-stakes applications in Machine-Learning-as-a-Service (ML)<n>Still, their black-box deployment exposes them to Model Extraction (ME) attacks.<n>Existing defenses suffer from a critical ''Euclidean bias''<n>We propose GraphRP, a proactive defense framework that repurposes Model Reprogramming for security.
  arXiv  Detail & Related papers  (2026-08-11T23:20:15Z)
• SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMs [74.89208610190595]
  We present SPOT-E, a test-time method that produces question-conditioned spotlights, optimized per instance via light-weight tuning.<n> SPOT-E yields consistent gains and improved robustness under visual corruptions.
  arXiv  Detail & Related papers  (2026-06-18T13:56:30Z)
• Not All Tokens Learn Alike: Attention Entropy Reveals Heterogeneous Signals in RL Reasoning [11.322734738973603]
  Reinforcement-learning-based post-training has become a key approach for improving the reasoning ability of large language models.<n>This work studies their heterogeneity through attention entropy, which measures how concentrated or diffuse the contextual support is for each response token.
  arXiv  Detail & Related papers  (2026-05-08T12:31:28Z)
• Learning Uncertainty from Sequential Internal Dispersion in Large Language Models [52.29267172760918]
  Uncertainty estimation is a promising approach to detect hallucinations in large language models.<n>Recent approaches commonly depend on model internal states to estimate uncertainty.<n>We present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework.
  arXiv  Detail & Related papers  (2026-04-17T06:31:29Z)
• Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models [54.16797570104461]
  Diffusion-based language models (dLLMs) have emerged as a promising alternative to autoregressive language models.<n>We investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis.
  arXiv  Detail & Related papers  (2026-04-12T10:26:41Z)
• DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation [22.14448091276763]
  A common strategy for security hardening is to fine-tune models using supervision derived from the final transformer layer.<n>This design may suffer from a final-layer bottleneck: vulnerability-discriminative cues can be distributed across layers and become less detectable near the output representations optimized for next-token prediction.<n>We introduce DeepGuard, a framework that leverages distributed security-relevant cues by aggregating representations from multiple upper layers via an attention-based module.
  arXiv  Detail & Related papers  (2026-04-10T08:19:48Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover [30.86966284669791]
  Adrial attacks can reliably steer safety-aligned large language models toward unsafe behavior.<n>We propose a theoretical generative model of proxy language in terms of a spin-glass system operating in a replica-symmetry-breaking regime.<n>Within this framework, we analyze prompt injection-based jailbreaking.
  arXiv  Detail & Related papers  (2026-03-11T21:48:03Z)
• Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models [2.6140509675507384]
  We study jailbreaking from both security and interpretability perspectives.<n>We propose a tensor-based latent representation framework that captures structure in hidden activations.<n>Our results provide evidence that jailbreak behavior is rooted in identifiable internal structures.
  arXiv  Detail & Related papers  (2026-02-12T02:43:17Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• Mechanistic Interpretability in the Presence of Architectural Obfuscation [0.0]
  Architectural obfuscation is a lightweight substitute for heavyweight cryptography in privacy-preserving large-language-model (LLM) inference.<n>We analyze a GPT-2-small model trained from scratch with a representative obfuscation map.<n>Our findings reveal that obfuscation dramatically alters activation patterns within attention heads yet preserves the layer-wise computational graph.
  arXiv  Detail & Related papers  (2025-06-22T14:39:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.