Trojan Hippo: Weaponizing Agent Memory for Data Exfiltration

• URL: http://arxiv.org/abs/2605.01970v2
• Date: Tue, 05 May 2026 11:52:20 GMT
• Title: Trojan Hippo: Weaponizing Agent Memory for Data Exfiltration
• Abstract summary: Trojan Hippo is a class of persistent memory attacks that operates in a more realistic threat model than prior memory poisoning work.<n>We evaluate four memory-system defenses inspired by basic security principles, finding they substantially reduce attack success rates.<n>Because of this substantial security-utility tradeoff, the effective real-world deployment of defenses remains an open challenge.
• Score: 33.8989871605613
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Memory systems enable otherwise-stateless LLM agents to persist user information across sessions, but also introduce a new attack surface. We characterize the Trojan Hippo attack, a class of persistent memory attacks that operates in a more realistic threat model than prior memory poisoning work: the attacker plants a dormant payload into an agent's long-term memory via a single untrusted tool call (e.g., a crafted email), which activates only when the user later discusses sensitive topics such as finance, health, or identity, and exfiltrates high-value personal data to the attacker. While anecdotal demonstrations of such attacks have appeared against deployed systems, no prior work systematically evaluates them across heterogeneous memory architectures and defenses. We introduce a dynamic evaluation framework comprising two components: (1) an OpenEvolve-based adaptive red-teaming benchmark that stress-tests defenses and memory backends against continuously refined attacks, and (2) the first capability-aware security/utility analysis for persistent memory systems, enabling principled reasoning about defense deployment across different usage profiles. Instantiated on an email assistant across four memory backends (explicit tool memory, agentic memory, RAG, and sliding-window context), Trojan Hippo achieves up to 85-100% ASR against current frontier models from OpenAI and Google, with planted memories successfully activating even after 100 benign sessions. We evaluate four memory-system defenses inspired by basic security principles, finding they substantially reduce attack success rates (to as low as 0-5%), though at utility costs that vary widely with task requirements. Because of this substantial security-utility tradeoff, the effective real-world deployment of defenses remains an open challenge, which our evaluation framework is specifically designed to address.

Related papers

• A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks [53.12252890906191]
  Large language models (LLMs) remain vulnerable to jailbreak attacks.<n>We propose a self-evolving test-time defense built around a persistent, cross-interaction rule memory.
  arXiv  Detail & Related papers  (2026-08-26T16:52:06Z)
• Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems [11.238342775567856]
  We study prompt injection attacks in memory-based agentic systems using a sandboxed synthetic workspace.<n>Our results show that although it is difficult to make an agent overwrite its own memory files using untrusted external content, payloads already planted in those files can successfully attack current and future sessions.
  arXiv  Detail & Related papers  (2026-07-16T06:13:48Z)
• When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agents [55.87577638514179]
  We study the threat as stealth memory injection, in which a remote black-box delivers a single email payload that must induce the agent to poisoned memory.<n>We propose MemGhost, a one-shot payload generation framework.<n>Across 56 held-out test cases, MemGhost achieves 87.5% end-to-end success on OpenClaw with GPT-5.4 and 71.4% on Claude Code SDK with Sonnet 4.6.
  arXiv  Detail & Related papers  (2026-07-06T15:08:58Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents [74.64265314956441]
  We propose a unified black-box attack framework that poisons graph-structured external memory with coordinated text-image evidence.<n>MemVenom achieves strong end-to-end attack success with minimal impact on benign performance, reaching up to 99.15% on GPT-5-family web agents.
  arXiv  Detail & Related papers  (2026-06-09T11:53:25Z)
• Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interaction [3.809725488301918]
  Large language model (LLM) agents increasingly leverage long term memory to support persistent and autonomous task execution.<n>Existing memory poisoning attacks assume that injected content can be stored directly in memory, overlooking the selective extraction and rewriting stages.<n>Mechanistic analysis indicates that the attack exploits embedding-space anisotropy and shifts attention patterns.
  arXiv  Detail & Related papers  (2026-05-28T14:02:00Z)
• Adaptive Defense Orchestration for RAG: A Sentinel-Strategist Architecture against Multi-Vector Attacks [0.0]
  Retrieval-augmented generation systems are increasingly deployed in sensitive domains such as healthcare and law.<n>This capability introduces significant security risks, including membership inference, data poisoning, and unintended content leakage.<n>We propose the Sentinel-Strategist architecture, a context-aware framework for risk analysis and defense selection.
  arXiv  Detail & Related papers  (2026-04-22T11:17:10Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections [57.64370755825839]
  Self-evolving agents update their internal state across sessions, often by writing and reusing long-term memory.<n>We study this risk and formalize a persistent attack we call a Zombie Agent.<n>We present a black-box attack framework that uses only indirect exposure through attacker-controlled web content.
  arXiv  Detail & Related papers  (2026-02-17T15:28:24Z)
• Memory Poisoning Attack and Defense on Memory Based LLM-Agents [3.7127635602605014]
  Large language model agents equipped with persistent memory are vulnerable to memory poisoning attacks.<n>Recent work demonstrated that the MINJA (Memory Injection Attack) achieves over 95 % injection success rate.<n>This work addresses gaps through systematic empirical evaluation of memory poisoning attacks and defenses.
  arXiv  Detail & Related papers  (2026-01-09T03:26:10Z)
• A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory [31.673865459672285]
  Large Language Model (LLM) agents use memory to learn from past interactions.<n>An adversary can inject seemingly harmless records into an agent's memory to manipulate its future behavior.<n>A-MemGuard is the first proactive defense framework for LLM agent memory.
  arXiv  Detail & Related papers  (2025-09-29T16:04:15Z)
• Cuckoo Attack: Stealthy and Persistent Attacks Against AI-IDE [64.47951172662745]
  Cuckoo Attack is a novel attack that achieves stealthy and persistent command execution by embedding malicious payloads into configuration files.<n>We formalize our attack paradigm into two stages, including initial infection and persistence.<n>We contribute seven actionable checkpoints for vendors to evaluate their product security.
  arXiv  Detail & Related papers  (2025-09-19T04:10:52Z)
• Memory Under Siege: A Comprehensive Survey of Side-Channel Attacks on Memory [0.210674772139335]
  Side-channel attacks on memory (SCAM) exploit unintended data leaks from memory subsystems to infer sensitive information.<n>This research aims to examine SCAM, classify various attack techniques, and evaluate existing defense mechanisms.
  arXiv  Detail & Related papers  (2025-05-08T02:16:08Z)
• Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense [90.71884758066042]
  Large vision-language models (LVLMs) introduce a unique vulnerability: susceptibility to malicious attacks via visual inputs.<n>We propose ESIII (Embedding Security Instructions Into Images), a novel methodology for transforming the visual space from a source of vulnerability into an active defense mechanism.
  arXiv  Detail & Related papers  (2025-03-14T17:39:45Z)
• AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases [73.04652687616286]
  We propose AgentPoison, the first backdoor attack targeting generic and RAG-based LLM agents by poisoning their long-term memory or RAG knowledge base. Unlike conventional backdoor attacks, AgentPoison requires no additional model training or fine-tuning. On each agent, AgentPoison achieves an average attack success rate higher than 80% with minimal impact on benign performance.
  arXiv  Detail & Related papers  (2024-07-17T17:59:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.