Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense

• URL: http://arxiv.org/abs/2606.29441v1
• Date: Sun, 28 Jun 2026 15:05:12 GMT
• Title: Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense
• Abstract summary: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.<n>We evaluate five defense paradigms (no defense, static steering, CAST, AlphaSteer, probe-gated) across seven instruction-tuned models.<n>AlphaSteer achieves 0% attack success on GCG, AutoDAN, and intent laundering but 50% on prefilling.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists. We evaluate five defense paradigms (no defense, static steering, CAST, AlphaSteer, probe-gated) across seven instruction-tuned models (7-31B) and five attack types (GCG, AutoDAN, DeepInception, prefilling, intent laundering). Our central finding: prompt-time activation defenses are structurally blind to prefilling attacks. AlphaSteer achieves 0% attack success on GCG, AutoDAN, and intent laundering but 50% on prefilling. We prove a corollary: any defense that gates intervention on a single layer's activation alignment with a benign reference (cone, subspace, or null-space) is blind to attacks that craft activations to lie inside that reference, whether checked at prompt time or per token. As its constructive contrapositive we introduce response-time probing: a linear probe on the model's hidden state at the first generated tokens, with AUROC 0.97-1.00 across all seven models. Combined with a halt, it cuts prefilling attack success to 0/40 on every model with 0% benign false positives, outperforming Llama Guard 3. Cross-template generalisation depends on probe depth, so we scope the claim to the canonical prefilling-template family. Composing the response-halt with AlphaSteer's null-space steering gives an orthogonal split (the halt catches prefilling, AlphaSteer catches semantic attacks), reaching defense success 0.983 on Mistral and 0.994 on Llama and dominating both components. We further show MMLU fails to capture steering's true utility cost, which appears as behavioral hedging rather than factual loss, and that diverse negative training sets cut probe false positives from 80-100% to near zero. Code, attacks, per-sample results, and the judge prompt are released.

Related papers

• Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models [4.1626636325601405]
  Our defense, decoy hardening, concedes the refusal strip and poisons its payoff.<n>We instantiate it on seven models from five families (9B-122B, dense and mixture-of-experts)
  arXiv  Detail & Related papers  (2026-08-17T23:26:32Z)
• A New Kind of Adversarial Example: Measuring the Human-Model Gap, and Its Relationship to OOD Detection [18.313155919803034]
  A large, clearly visible perturbation causes the model to keep its original, correct prediction, even though a human would no longer recognize the image.<n>We answer whether humans really perform worse than the model, whether standard out-of-distribution (OOD) detection and calibration tools catch it, and whether existing defenses mitigate it.
  arXiv  Detail & Related papers  (2026-07-22T00:44:06Z)
• Comparative Analysis of Inference-Time Defense Methods for Multimodal Large Language Models [0.0]
  Multimodal large language models (MLLMs) now appear in safety-critical applications.<n>Retraining a model for each new vulnerability class is usually too expensive to be practical.<n>We report a comparative empirical evaluation of three-time defense methods and their combinations.
  arXiv  Detail & Related papers  (2026-06-09T14:13:54Z)
• Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack [78.02110947708535]
  Adversarial images pose a severe security threat to multimodal large language models through prompt injection.<n>We show that successful adversarial attacks do not rely on the entire image uniformly but instead depend on a small subset of critical image tokens.<n>We propose Gradient Token Masking (GTM), which localizes these tokens via gradient analysis and neutralizes them through masking.
  arXiv  Detail & Related papers  (2026-05-24T17:51:34Z)
• The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems [1.0262304700896199]
  We formalize emphSemantic Norm Drift (SND) as a third path to agent misconduct.<n>In SND, a policy-formatted document enters a shared vector store through normal uploads and later reappears as trusted system context.<n>Counterfactual Composition Testing identifies the causal entry with 87.5% accuracy and zero false positives.
  arXiv  Detail & Related papers  (2026-05-12T20:21:47Z)
• Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoning [1.3011345529764784]
  A new generation of language models reasons entirely in continuous hidden states, producing no tokens and leaving no audit trail.<n>ThoughtSteer achieves >=99% attack success rate with near-baseline clean accuracy.<n>Individual latent vectors still encode the correct answer even as the model outputs the wrong one.
  arXiv  Detail & Related papers  (2026-04-01T11:34:55Z)
• TrapSuffix: Proactive Defense Against Adversarial Suffixes in Jailbreaking [52.72486831074384]
  Suffix-based jailbreak attacks append an adversarial suffix, i.e., a short token sequence, to steer aligned LLMs into unsafe outputs.<n>We propose TrapSuffix, a lightweight fine-tuning approach that injects trap-aligned behaviors into the base model without changing the inference pipeline.<n>Across diverse suffix-based jailbreak settings, TrapSuffix reduces the average attack success rate to below 0.01 percent and achieves an average tracing success rate of 87.9 percent.
  arXiv  Detail & Related papers  (2026-02-06T11:43:56Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Checkpoint-GCG: Auditing and Attacking Fine-Tuning-Based Prompt Injection Defenses [10.08464073347558]
  We introduce Checkpoint-GCG, a white-box attack against fine-tuning-based defenses.<n>We show Checkpoint-GCG to achieve up to $96%$ attack success rate (ASR) against the strongest defense.
  arXiv  Detail & Related papers  (2025-05-21T16:43:17Z)
• Fundamental Limitations in Pointwise Defences of LLM Finetuning APIs [56.84338097581665]
  We show that defences of fine-tuning APIs are fundamentally limited in their ability to prevent fine-tuning attacks.<n>We construct 'pointwise-undetectable' attacks that repurpose entropy in benign model outputs to covertly transmit dangerous knowledge.<n>We test our attacks against the OpenAI fine-tuning API, finding they succeed in eliciting answers to harmful multiple-choice questions.
  arXiv  Detail & Related papers  (2025-02-20T18:45:01Z)
• Evaluating the Robustness of the "Ensemble Everything Everywhere" Defense [90.7494670101357]
  Ensemble everything everywhere is a defense to adversarial examples.<n>We show that this defense is not robust to adversarial attack.<n>We then use standard adaptive attack techniques to reduce the defense's robust accuracy.
  arXiv  Detail & Related papers  (2024-11-22T10:17:32Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning [95.60856995067083]
  This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. We propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%.
  arXiv  Detail & Related papers  (2021-06-01T07:10:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.