Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Model Safety Tiers

• URL: http://arxiv.org/abs/2603.28013v2
• Date: Fri, 03 Apr 2026 17:43:16 GMT
• Title: Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Model Safety Tiers
• Abstract summary: We present a stage-decomposed analysis of prompt injection attacks against five frontier LLM agents.<n>We instrument every run with a cryptographic canary token (SECRET-[A-F0-9]8) tracked through four kill-chain stages.<n>Our central finding is that model safety is determined not by whether adversarial content is seen, but by whether it is propagated across pipeline stages.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present a stage-decomposed analysis of prompt injection attacks against five frontier LLM agents. Prior work measures task-level attack success rate (ASR); we localize the pipeline stage at which each model's defense activates. We instrument every run with a cryptographic canary token (SECRET-[A-F0-9]{8}) tracked through four kill-chain stages -- Exposed, Persisted, Relayed, Executed -- across four attack surfaces and five defense conditions (764 total runs, 428 no-defense attacked). Our central finding is that model safety is determined not by whether adversarial content is seen, but by whether it is propagated across pipeline stages. Concretely: (1) in our evaluation, exposure is 100% for all five models -- the safety gap is entirely downstream; (2) Claude strips injections at write_memory summarization (0/164 ASR), while GPT-4o-mini propagates canaries without loss (53% ASR, 95% CI: 41--65%); (3) DeepSeek exhibits 0% ASR on memory surfaces and 100% ASR on tool-stream surfaces from the same model -- a complete reversal across injection channels; (4) all four active defense conditions (write_filter, pi_detector, spotlighting, and their combination) produce 100% ASR due to threat-model surface mismatch; (5) a Claude relay node decontaminates downstream agents -- 0/40 canaries survived into shared memory.

Related papers

• AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories [0.0]
  LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter.<n>Existing benchmarks measure whether such attacks succeed against live agents, and existing guard models judge a trace as a whole.<n>We present AgentDrift, a benchmark of 12,536 synthetic tool-call trajectories over five agent domains.
  arXiv  Detail & Related papers  (2026-09-07T03:10:23Z)
• SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models [73.04773649437375]
  Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior.<n>We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale.<n>We also construct SafeAtlas-Bench, a set of 5,000 instances for evaluating five-level predictions and continuous risk scores.
  arXiv  Detail & Related papers  (2026-08-29T07:08:01Z)
• RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots [0.0]
  We present the first systematic multi-channel empirical study of prompt injection attacks delivered through the sensory pipeline of a ROS 2-based robotic system.<n>We identify model-specific vulnerability profiles that do not follow a monotonic scaling trend.<n>We propose a hybrid semantic firewall that achieves 0% ASR against known injection patterns with no false positives.
  arXiv  Detail & Related papers  (2026-06-26T23:31:59Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• Depth-Dependent Indirect Prompt Injection in Tool-Calling ReAct Agents: Injection Depth, Payload Framing, and Turn-Budget Sensitivity [0.0]
  We conduct 460 trials against GPT-4o-mini and Claude Haiku at a combined API cost under 0.36 USD.<n>Study 1 shows that ASR against GPT-4o-mini decays from 60% at depth 1 to 0% at 4 and 5.<n>Study 2 replicates the depth experiment on Claude Haiku, which achieves 0% ASR at every depth through a combination of conservative tool invocation and genuine instruction resistance.<n>Study 4 confirms that ASR is stable across turn caps of 3, 5, and 7, indicating the turn budget is not a risk factor in this setting.
  arXiv  Detail & Related papers  (2026-05-29T00:28:42Z)
• PlanFlip: Attacking Multi-Agent LLM Systems via Planning-Phase Prompt Injection [19.162716475205308]
  Multi-agent LLM systems increasingly rely on a Planner to decompose goals into sub-task sequences.<n>A single injection into the Planner's context achieves cascade amplification, corrupting all downstream sub-tasks simultaneously.<n>We introduce PlanFlip, a framework comprising four planning-phase prompt injection attacks.
  arXiv  Detail & Related papers  (2026-04-30T07:22:06Z)
• SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Do [0.0]
  SafetyDrift models agent safety trajectories as absorbing Markov chains.<n>In communication tasks, agents that reach even a mild risk state have an 85% chance of violating safety within five steps.<n>A monitor built on these models detects 94.7% of violations with 3.7 steps of advance warning at negligible computational cost.
  arXiv  Detail & Related papers  (2026-03-28T05:52:04Z)
• VisualLeakBench: Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering [14.756677328512907]
  VisualLeakBench is an evaluation suite to audit LVLMs against OCR Injection and Contextual PII Leakage.<n>We use 1,000 synthetically generated adversarial images with 8 PII types, validated on 50 in-the-wild (IRL) real-world screenshots.<n>We release our dataset and code for reproducible robustness and safety evaluation of deployment-relevant vision-language systems.
  arXiv  Detail & Related papers  (2026-03-11T05:47:24Z)
• Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems [0.0]
  Retrieval-Augmented Generation (RAG) systems extend large language models with external knowledge sources.<n>In particular, adversaries can poison retrieval corpora so that malicious documents are preferentially retrieved at inference time.<n>We study gradient-guided corpus poisoning attacks against modern RAG pipelines and evaluate retrieval-layer defenses.
  arXiv  Detail & Related papers  (2026-03-10T23:15:13Z)
• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models [54.08784776767683]
  We present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels.<n>PhysCond-WMA induces semantic, logic, or decision-level distortion while preserving perceptual fidelity.
  arXiv  Detail & Related papers  (2026-02-21T07:22:37Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• STACK: Adversarial Attacks on LLM Safeguard Pipelines [5.784929232265091]
  Anthropic guards latest Claude 4 Opus model using one such defense pipeline.<n>Other frontier developers including Google DeepMind and OpenAI pledge to soon deploy similar defenses.<n>We address this gap by developing and red-teaming an open-source defense pipeline.
  arXiv  Detail & Related papers  (2025-06-30T17:21:08Z)
• Evaluating the Robustness of the "Ensemble Everything Everywhere" Defense [90.7494670101357]
  Ensemble everything everywhere is a defense to adversarial examples.<n>We show that this defense is not robust to adversarial attack.<n>We then use standard adaptive attack techniques to reduce the defense's robust accuracy.
  arXiv  Detail & Related papers  (2024-11-22T10:17:32Z)
• BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models [57.5404308854535]
  Safety backdoor attacks in large language models (LLMs) enable the stealthy triggering of unsafe behaviors while evading detection during normal interactions. We present BEEAR, a mitigation approach leveraging the insight that backdoor triggers induce relatively uniform drifts in the model's embedding space. Our bi-level optimization method identifies universal embedding perturbations that elicit unwanted behaviors and adjusts the model parameters to reinforce safe behaviors against these perturbations.
  arXiv  Detail & Related papers  (2024-06-24T19:29:47Z)
• FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning [66.56240101249803]
  We study how hardening benign clients can affect the global model (and the malicious clients) We propose a trigger reverse engineering based defense and show that our method can achieve improvement with guarantee robustness. Our results on eight competing SOTA defense methods show the empirical superiority of our method on both single-shot and continuous FL backdoor attacks.
  arXiv  Detail & Related papers  (2022-10-23T22:24:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.