Poisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content

• URL: http://arxiv.org/abs/2605.24421v1
• Date: Sat, 23 May 2026 06:21:10 GMT
• Title: Poisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content
• Abstract summary: Large language models (LLMs) are increasingly used as analyst assistants in security operations centers (SOCs)<n>We study a structural failure mode of this design: many log fields are attacker controlled.<n>We introduce a four-class taxonomy of log-substrate attacks: direct override (S1), persona hijack (S2), context manipulation (S3), and obfuscated payloads (S4)
• Score: 1.5369106213673014
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly used as analyst assistants in security operations centers (SOCs), where they ingest log and alert data to produce triage labels, incident summaries, or remediation advice. We study a structural failure mode of this design: many log fields are attacker controlled. User agents, URLs, payloads, DNS queries, and attempted usernames can therefore carry instructions to the model alongside evidence of the intrusion. We call this setting \emph{log-substrate prompt injection}. We introduce a four-class taxonomy of log-substrate attacks: direct override (S1), persona hijack (S2), context manipulation (S3), and obfuscated payloads (S4). We evaluate 48 strategy-defense-task combinations using \texttt{gpt-4o-mini} as the analyst. Three findings stand out. First, direct overrides are ineffective in our setting: all S1 classification attacks achieve 0\% suppression. In contrast, persona hijacks suppress 68\% of malicious logs under a naive classifier and remain effective under stronger defenses. Second, summarization is the highest-risk task: context manipulation reaches 96\% injection success without defenses and 38\% even with constrained output. Third, defenses reduce but do not eliminate the attack surface: average injection success falls from 26.6\% under naive prompting to 11.8\% under our strongest defense. We also compare empirical results to a deterministic mock analyst and find that simulation substantially mispredicts current model behavior, especially for direct overrides. These results suggest that SOC copilots should treat raw log content as adversarial input rather than ordinary analyst context.

Related papers

• Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents [0.0]
  Recent work has converged on a strategy for defending tool-using LLM agents against indirect prompt injection.<n>We organize these out-of-band defenses as instances of classical integrity protection.<n>We run that protocol as an independent reproduction and extension of Progent's own adaptive-attack analysis.
  arXiv  Detail & Related papers  (2026-06-25T00:35:23Z)
• Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment [25.752599132396437]
  Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.<n>Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.<n>We propose RETA, a training-based method that grounds defense decisions on the user tasks rather than attacker-controlled data.
  arXiv  Detail & Related papers  (2026-06-13T19:15:44Z)
• AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents [57.34566159148893]
  Indirect prompt injection (IPI) is a major security threat to LLM-powered agents.<n>We develop AutoDojo, an adaptive extension of AgentDojo that optimize IPI against a given defense.
  arXiv  Detail & Related papers  (2026-06-13T02:09:08Z)
• Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning [0.9236074230806578]
  Oracle Poisoning is an attack class in which an adversary corrupts a structured knowledge graph.<n>Unlike prompt injection, Oracle Poisoning manipulates the data agents reason over, not their instructions.<n>We demonstrate six attack scenarios against a production 42-million-node code knowledge graph.
  arXiv  Detail & Related papers  (2026-05-10T23:55:13Z)
• Defenses Against Prompt Attacks Learn Surface Heuristics [40.392588465939106]
  Large language models (LLMs) are increasingly deployed in security-sensitive applications.<n>LLMs may override intended logic when adversarial instructions appear in user queries or retrieved content.<n>Recent defenses rely on supervised fine-tuning with benign and malicious labels.
  arXiv  Detail & Related papers  (2026-01-12T04:12:48Z)
• Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems [7.15710884787427]
  Large language models (LLMs) increasingly rely on retrieving information from external corpora.<n>This creates a new attack surface: indirect prompt injection (IPI)<n>We present the first end-to-end IPI exploits under natural queries and realistic external corpora.
  arXiv  Detail & Related papers  (2026-01-11T21:33:59Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis [48.70474961584997]
  Indirect prompt injection attacks (IPIAs) pose a critical threat to large language models (LLMs)<n>We present IntentGuard, a general defense framework based on instruction-following intent analysis.
  arXiv  Detail & Related papers  (2025-11-30T16:29:04Z)
• Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain [82.98626829232899]
  Fine-tuning AI agents on data from their own interactions introduces a critical security vulnerability within the AI supply chain.<n>We show that adversaries can easily poison the data collection pipeline to embed hard-to-detect backdoors.
  arXiv  Detail & Related papers  (2025-10-03T12:47:21Z)
• Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents [54.35629963816521]
  This work introduces VIBMA, the first clean-text backdoor attack targeting VLM-based mobile agents.<n>The attack injects malicious behaviors into the model by modifying only the visual input.<n>We show that our attack achieves high success rates while preserving clean-task behavior.
  arXiv  Detail & Related papers  (2025-06-16T08:09:32Z)
• MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents [60.30753230776882]
  LLM agents are vulnerable to indirect prompt injection (IPI) attacks, where malicious tasks embedded in tool-retrieved information can redirect the agent to take unauthorized actions.<n>We present MELON, a novel IPI defense that detects attacks by re-executing the agent's trajectory with a masked user prompt modified through a masking function.
  arXiv  Detail & Related papers  (2025-02-07T18:57:49Z)
• InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models [7.186499635424984]
  Prompt injection attacks pose a critical threat to large language models (LLMs)<n> Prompt guard models, though effective in defense, suffer from over-defense due to trigger word bias.<n>InjecGuard is a novel prompt guard model that incorporates a new training strategy, Mitigating Over-defense for Free.
  arXiv  Detail & Related papers  (2024-10-30T07:39:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.