Latent Adversarial Detection: Adaptive Probing of LLM Activations for Multi-Turn Attack Detection

• URL: http://arxiv.org/abs/2604.28129v1
• Date: Thu, 30 Apr 2026 17:16:33 GMT
• Title: Latent Adversarial Detection: Adaptive Probing of LLM Activations for Multi-Turn Attack Detection
• Abstract summary: Multi-turn prompt injection follows a known attack path.<n>We show this attack path leaves an activation-level signature in the model's residual stream.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multi-turn prompt injection follows a known attack path -- trust-building, pivoting, escalation but text-level defenses miss covert attacks where individual turns appear benign. We show this attack path leaves an activation-level signature in the model's residual stream: each phase shift moves the activation, producing a total path length far exceeding benign conversations. We call this adversarial restlessness. Five scalar trajectory features capturing this signal lift conversation-level detection from 76.2% to 93.8% on synthetic held-out data. The signal replicates across four model families (24B-70B); probes are model-specific and do not transfer across architectures. Generalization is source-dependent: leave-one-source-out evaluation shows each of synthetic, LMSYS-Chat-1M, and SafeDialBench captures distinct attack distributions, with detection on real-world LMSYS reaching 47-71% when its distribution is represented in training. Combined three-source training achieves 89.4% detection at 2.4% false positive rate on a held-out mixed set. We further show that three-phase turn-level labels(benign/pivoting/adversarial) unique to our synthetic dataset are essential: binary conversation-level labels produce 50-59% false positives. These results establish adversarial restlessness as a reliable activation-level signal and characterize the data requirements for practical deployment.

Related papers

• HalluTracer: Hallucination Detection via Depth-Averaging Truth Signals [45.64606191137958]
  We introduce HalluTracer, a framework that aggregates truthfulness evidence across every layer of the forward pass.<n>A geometric analysis reveals that the per-layer signals are weakly correlated, so that simple depth averaging suppresses layer-specific noise.<n>HalluTracer consistently outperforms matched white-box baselines.
  arXiv  Detail & Related papers  (2026-08-17T10:03:29Z)
• TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs [5.676157325165344]
  Large language models (LLMs) can generate fluent reasoning traces that nevertheless lead to incorrect answers.<n>TrAC is a correctness-supervised uncertainty quantification framework that combines active and passive signals anchored to one completed reasoning trace.<n>TrAC improves macro AUROC by 1.8% and reduces AURC by 3.4% relative to eight-sample self-consistency.
  arXiv  Detail & Related papers  (2026-08-01T03:43:31Z)
• RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots [0.0]
  We present the first systematic multi-channel empirical study of prompt injection attacks delivered through the sensory pipeline of a ROS 2-based robotic system.<n>We identify model-specific vulnerability profiles that do not follow a monotonic scaling trend.<n>We propose a hybrid semantic firewall that achieves 0% ASR against known injection patterns with no false positives.
  arXiv  Detail & Related papers  (2026-06-26T23:31:59Z)
• Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agents [1.0152838128195467]
  We show that Evaluator Preference Collapse ( EPC) is dramatically amplified in multimodal settings.<n>We then demonstrate a novel phenomenon we term cross-modal contagion.<n>We identify cross-model evaluator architecture as the primary risk factor for preference drift.
  arXiv  Detail & Related papers  (2026-06-15T13:18:20Z)
• Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs [52.149036302760386]
  Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs.<n>Recent work proposes detection using linear probes that recover the secret from internal activations.<n>We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention.
  arXiv  Detail & Related papers  (2026-06-08T12:27:11Z)
• Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories [0.0]
  LLM-based coding agents sometimes acknowledge a problem in their own reasoning and then proceed anyway.<n>We build a Claude Sonnet 4.6 judge that reads full trajectories and flags spans where the pattern occurs.<n>We evaluate it on 44 Terminal-bench-2 trajectories using a Qwen3.5-35B-A3B backbone.
  arXiv  Detail & Related papers  (2026-06-05T22:52:16Z)
• In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification [4.738949927143789]
  We show that homogeneous labels--even semantically valid ones--collapse accuracy to =12% across six models.<n>The trigger is label-slot content: the model treats tokens occupying the label position as an exhaustive answer vocabulary.
  arXiv  Detail & Related papers  (2026-05-08T10:20:39Z)
• Poisoned Acoustics [0.0]
  Training-data poisoning attacks can induce targeted, undetectable failure in deep neural networks by corrupting a vanishingly small fraction of training labels.<n>We demonstrate this on acoustic vehicle classification using the MELAUDIS urban intersection dataset.
  arXiv  Detail & Related papers  (2026-02-25T01:09:43Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• Peak + Accumulation: A Proxy-Level Scoring Formula for Multi-Turn LLM Attack Detection [0.0]
  Multi-turn prompt injection attacks distribute malicious intent across multiple conversation turns.<n>No published formula exists for aggregating per-turn pattern scores into a conversation-level risk score at the proxy layer.<n>We propose peak + accumulation scoring -- a formula combining peak single-turn risk, persistence ratio, and category diversity.
  arXiv  Detail & Related papers  (2026-02-11T17:53:41Z)
• Mitigating Spurious Correlations in NLI via LLM-Synthesized Counterfactuals and Dynamic Balanced Sampling [0.0]
  Natural Language Inference (NLI) models frequently rely on spurious correlations rather than semantic reasoning.<n>Existing mitigation strategies often incur high annotation costs or trigger catastrophic forgetting during fine-tuning.<n>We propose an automated, scalable pipeline to address these limitations.
  arXiv  Detail & Related papers  (2025-12-20T18:30:54Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models [57.474294329887236]
  Diffusion large language models (dLLMs) generate text through iterative denoising.<n>Current decoding strategies discard rich intermediate predictions in favor of the final output.<n>We introduce two complementary methods that exploit temporal consistency.
  arXiv  Detail & Related papers  (2025-08-12T17:59:57Z)
• Backdoor Cleaning without External Guidance in MLLM Fine-tuning [76.82121084745785]
  Believe Your Eyes (BYE) is a data filtering framework that leverages attention entropy patterns as self-supervised signals to identify and filter backdoor samples.<n>It achieves near-zero attack success rates while maintaining clean-task performance.
  arXiv  Detail & Related papers  (2025-05-22T17:11:58Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Weakly Supervised Veracity Classification with LLM-Predicted Credibility Signals [4.895830603263421]
  Pastel is a weakly supervised approach that leverages large language models to extract credibility signals from web content. We study the association between credibility signals and veracity, and perform a study showing the impact of each signal on model performance.
  arXiv  Detail & Related papers  (2023-09-14T11:06:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.