Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning

• URL: http://arxiv.org/abs/2605.09822v1
• Date: Sun, 10 May 2026 23:55:13 GMT
• Title: Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning
• Abstract summary: Oracle Poisoning is an attack class in which an adversary corrupts a structured knowledge graph.<n>Unlike prompt injection, Oracle Poisoning manipulates the data agents reason over, not their instructions.<n>We demonstrate six attack scenarios against a production 42-million-node code knowledge graph.
• Score: 0.9236074230806578
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We define Oracle Poisoning, an attack class in which an adversary corrupts a structured knowledge graph that AI agents query at runtime via tool-use protocols, causing incorrect conclusions through correct reasoning. Unlike prompt injection, Oracle Poisoning manipulates the data agents reason over, not their instructions. We demonstrate six attack scenarios against a production 42-million-node code knowledge graph, providing the first empirical demonstration of knowledge graph poisoning against a production-scale agentic system, distinct from CTI embedding poisoning. Primary evaluation uses real SDK tool-use across nine models from three providers (N=30 per model), where models autonomously invoke a graph query tool and reason from results. The result is unambiguous: every tested model trusts poisoned data at 100% at moderate attacker sophistication(L2), with 269 valid trials (of 270) accepting fabricated security claims under directed queries. Under open-ended prompts, trust drops to 3-55%, confirming prompt framing as a confound; we report both conditions. An attacker sophistication gradient reveals discrete break points, a minimum skill at which trust flips from 0% to 100%, reframing the attack as a question not of whether but of how much. A controlled delivery-mode comparison shows that inline evaluation produces false negatives: GPT-5.1 shows 0% trust inline but 100% under both simulated and real agentic tool-use, demonstrating that delivery mode is a first-order confound. We evaluate five defences; read-only access control eliminates the direct mutation vector, while the remaining four are partial and model-dependent. Analysis of four additional platforms suggests the attack may generalise across the knowledge-graph ecosystem.

Related papers

• Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks [52.041551928780244]
  Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data.<n>We show that this can severely underestimate worst-case vulnerability.<n>We introduce SAILS (Set-level Audit-Informed Iterative Learned Selection), which learns a set scorer from a few hundred finetune-and-evaluate runs.
  arXiv  Detail & Related papers  (2026-09-14T04:44:28Z)
• SIR: Self-improving Red-teaming for Compute Use Agents [71.71987044117371]
  Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal.<n>Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI)<n>We present SIR, a black box IPI attack that composes stealthy injections from a small library of reusable principles stated in plain language.
  arXiv  Detail & Related papers  (2026-08-31T03:39:43Z)
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices [48.36933831982682]
  We introduce ADeptS-Bench, a trustworthiness benchmark grounded in the ADEPTS capability framework.<n>The Safety stream provides paired benign/malicious tasks with threats embedded in the visual interface.<n>The Disambiguation stream evaluates whether agents seek clarification when intent is ambiguous.
  arXiv  Detail & Related papers  (2026-08-25T23:21:57Z)
• Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agents [11.867868278947524]
  LLM-based agents are increasingly deployed in high-stakes scenarios such as email management, financial transactions, and code execution.<n>We propose AuthGraph, a dual-graph alignment defense framework that constructs two complementary graphs.<n>On AgentDojo, AuthGraph reduces the attack success rate from 40% to 1% while maintaining 76% task completion rate on GPT-4o; on AgentDyn, it reduces the attack success rate from 39% to 2% while preserving 51% utility.
  arXiv  Detail & Related papers  (2026-05-26T03:20:23Z)
• Poisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content [1.5369106213673014]
  Large language models (LLMs) are increasingly used as analyst assistants in security operations centers (SOCs)<n>We study a structural failure mode of this design: many log fields are attacker controlled.<n>We introduce a four-class taxonomy of log-substrate attacks: direct override (S1), persona hijack (S2), context manipulation (S3), and obfuscated payloads (S4)
  arXiv  Detail & Related papers  (2026-05-23T06:21:10Z)
• The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems [1.0262304700896199]
  We formalize emphSemantic Norm Drift (SND) as a third path to agent misconduct.<n>In SND, a policy-formatted document enters a shared vector store through normal uploads and later reappears as trusted system context.<n>Counterfactual Composition Testing identifies the causal entry with 87.5% accuracy and zero false positives.
  arXiv  Detail & Related papers  (2026-05-12T20:21:47Z)
• AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators [0.0]
  We introduce AgentCollabBench, a diagnostic benchmark of 900 human-validated tasks spanning software engineering, DevOps, and data engineering.<n>Each task isolates one of four behavioral risks: instruction decay, false-belief contagion, context leakage, and tracer durability.<n>We evaluate four modern LLMs (GPT 4.1 mini, Gemini 2.5 Flash Lite, Qwen-3.5-35B-A3B, and Llama 3.1 8B Instruct)<n> communication topology emerges as a primary risk factor that explains 7-40% of the variance in multi-hop information survival.
  arXiv  Detail & Related papers  (2026-05-09T03:35:09Z)
• Formal Architecture Descriptors as Navigation Primitives for AI Coding Agents [0.0]
  We investigate whether providing agents with formal architecture descriptors can reduce this navigational overhead.<n>We propose intent.lisp, an S-expression architecture descriptor, and open-source the Forge toolkit.
  arXiv  Detail & Related papers  (2026-04-11T00:26:31Z)
• Measuring and Exploiting Confirmation Bias in LLM-Assisted Security Code Review [6.417595678110472]
  We study whether confirmation bias affects LLM-based vulnerability detection, and whether this failure mode can be exploited in software supply-chain attacks.<n>Study 1 quantifies confirmation bias through controlled experiments on 250 CVE vulnerability/patch pairs evaluated across four state-of-the-art models under five framing conditions for the review prompt.<n>Study 2 evaluates exploitability in practice mimicking adversarial pull requests that reintroduce known vulnerabilities while framed as security improvements or urgent functionality fixes via their pull request metadata.
  arXiv  Detail & Related papers  (2026-03-19T10:40:27Z)
• The 'Sure' Trap: Multi-Scale Poisoning Analysis of Stealthy Compliance-Only Backdoors in Fine-Tuned Large Language Models [10.377264470934843]
  Backdoor attacks on large language models typically couple a secret trigger to an explicit malicious output.<n>We introduce a compliance-only backdoor: supervised fine-tuning on a mostly benign dataset in which a small subset of prompts is suffixed with an arbitrary single-word trigger.<n>We conduct a multi-scale analysis of this benign-label poisoning behavior across poison budget, total fine-tuning dataset size, and model size.
  arXiv  Detail & Related papers  (2025-11-16T02:01:58Z)
• Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain [82.98626829232899]
  Fine-tuning AI agents on data from their own interactions introduces a critical security vulnerability within the AI supply chain.<n>We show that adversaries can easily poison the data collection pipeline to embed hard-to-detect backdoors.
  arXiv  Detail & Related papers  (2025-10-03T12:47:21Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)
• Malicious Agent Detection for Robust Multi-Agent Collaborative Perception [52.261231738242266]
  Multi-agent collaborative (MAC) perception is more vulnerable to adversarial attacks than single-agent perception. We propose Malicious Agent Detection (MADE), a reactive defense specific to MAC perception. We conduct comprehensive evaluations on a benchmark 3D dataset V2X-sim and a real-road dataset DAIR-V2X.
  arXiv  Detail & Related papers  (2023-10-18T11:36:42Z)
• Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection [66.94175259287115]
  We propose a novel backdoor attack setting tailored for instruction-tuned LLMs. In a VPI attack, a backdoored model is expected to respond as if an attacker-specified virtual prompt were formalized to the user instruction. We demonstrate the threat by poisoning the model's instruction tuning data.
  arXiv  Detail & Related papers  (2023-07-31T17:56:00Z)
• IDEA: Invariant Defense for Graph Adversarial Robustness [60.0126873387533]
  We propose an Invariant causal DEfense method against adversarial Attacks (IDEA) We derive node-based and structure-based invariance objectives from an information-theoretic perspective. Experiments demonstrate that IDEA attains state-of-the-art defense performance under all five attacks on all five datasets.
  arXiv  Detail & Related papers  (2023-05-25T07:16:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.