Memory Injection Attacks on LLM Agents via Query-Only Interaction

• URL: http://arxiv.org/abs/2503.03704v3
• Date: Fri, 24 Oct 2025 19:47:39 GMT
• Title: Memory Injection Attacks on LLM Agents via Query-Only Interaction
• Authors: Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, Zhen Xiang,
• Abstract summary: We propose a novel Memory INJection Attack, MINJA, without assuming that the attacker can directly modify the memory bank of the agent.<n>The attacker injects malicious records into the memory bank by only interacting with the agent via queries and output observations.<n> MINJA enables any user to influence agent memory, highlighting the risk.
• Score: 49.14715983268449
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agents powered by large language models (LLMs) have demonstrated strong capabilities in a wide range of complex, real-world applications. However, LLM agents with a compromised memory bank may easily produce harmful outputs when the past records retrieved for demonstration are malicious. In this paper, we propose a novel Memory INJection Attack, MINJA, without assuming that the attacker can directly modify the memory bank of the agent. The attacker injects malicious records into the memory bank by only interacting with the agent via queries and output observations. These malicious records are designed to elicit a sequence of malicious reasoning steps corresponding to a different target query during the agent's execution of the victim user's query. Specifically, we introduce a sequence of bridging steps to link victim queries to the malicious reasoning steps. During the memory injection, we propose an indication prompt that guides the agent to autonomously generate similar bridging steps, with a progressive shortening strategy that gradually removes the indication prompt, such that the malicious record will be easily retrieved when processing later victim queries. Our extensive experiments across diverse agents demonstrate the effectiveness of MINJA in compromising agent memory. With minimal requirements for execution, MINJA enables any user to influence agent memory, highlighting the risk.

Related papers

• Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections [57.64370755825839]
  Self-evolving agents update their internal state across sessions, often by writing and reusing long-term memory.<n>We study this risk and formalize a persistent attack we call a Zombie Agent.<n>We present a black-box attack framework that uses only indirect exposure through attacker-controlled web content.
  arXiv  Detail & Related papers  (2026-02-17T15:28:24Z)
• AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management [47.49917373646469]
  Existing defenses treat bloated memory as given and focus on remaining resilient.<n>We present AgentSys, a framework that defends against indirect prompt injection through explicit memory management.
  arXiv  Detail & Related papers  (2026-02-07T06:28:51Z)
• MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval [5.734678752740074]
  MemoryGraft is a novel indirect injection attack that compromises agent behavior not through immediate jailbreaks, but by implanting malicious successful experiences into the agent's long-term memory.<n>We demonstrate that an attacker who can supply benign ingestion-level artifacts that the agent reads during execution can induce it to construct a poisoned RAG store.<n>When the agent later encounters semantically similar tasks, union retrieval over lexical templates and embedding similarity reliably surfaces these grafted memories, and the agent adopts the embedded unsafe patterns, leading to persistent behavioral drift across sessions.
  arXiv  Detail & Related papers  (2025-12-18T08:34:40Z)
• A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory [31.673865459672285]
  Large Language Model (LLM) agents use memory to learn from past interactions.<n>An adversary can inject seemingly harmless records into an agent's memory to manipulate its future behavior.<n>A-MemGuard is the first proactive defense framework for LLM agent memory.
  arXiv  Detail & Related papers  (2025-09-29T16:04:15Z)
• TopicAttack: An Indirect Prompt Injection Attack via Topic Transition [71.81906608221038]
  Large language models (LLMs) are vulnerable to indirect prompt injection attacks.<n>We propose TopicAttack, which prompts the LLM to generate a fabricated transition prompt that gradually shifts the topic toward the injected instruction.<n>We find that a higher injected-to-original attention ratio leads to a greater success probability, and our method achieves a much higher ratio than the baseline methods.
  arXiv  Detail & Related papers  (2025-07-18T06:23:31Z)
• Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution [7.2497315292753415]
  This paper examines how prompt injection can cause tool-calling agents to leak personal data observed during task execution.<n>Using a fictitious banking agent, we develop data flow-based attacks and integrate them into AgentDojo, a recent benchmark for agentic security.
  arXiv  Detail & Related papers  (2025-06-01T15:48:06Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction [68.6543680065379]
  Large language models (LLMs) are vulnerable to prompt injection attacks. We propose a novel defense method that leverages, rather than suppresses, the instruction-following abilities of LLMs.
  arXiv  Detail & Related papers  (2025-04-29T07:13:53Z)
• UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning [17.448966928905733]
  Large Language Model (LLM) agents equipped with external tools have become increasingly powerful for handling complex tasks.<n>We present UDora, a unified red teaming framework designed for LLM Agents that dynamically leverages the agent's own reasoning processes to compel it toward malicious behavior.
  arXiv  Detail & Related papers  (2025-02-28T21:30:28Z)
• Unveiling Privacy Risks in LLM Agent Memory [40.26158509307175]
  Large Language Model (LLM) agents have become increasingly prevalent across various real-world applications.<n>They enhance decision-making by storing private user-agent interactions in the memory module for demonstrations.<n>We propose a Memory EXTRaction Attack (MEXTRA) to extract private information from memory.
  arXiv  Detail & Related papers  (2025-02-17T19:55:53Z)
• MELON: Indirect Prompt Injection Defense via Masked Re-execution and Tool Comparison [60.30753230776882]
  LLM agents are vulnerable to indirect prompt injection (IPI) attacks.<n>We present MELON, a novel IPI defense.<n>We show that MELON outperforms SOTA defenses in both attack prevention and utility preservation.
  arXiv  Detail & Related papers  (2025-02-07T18:57:49Z)
• Towards Action Hijacking of Large Language Model-based Agent [39.19067800226033]
  We introduce Name, a novel hijacking attack to manipulate the action plans of black-box agent system.<n>Our approach achieved an average bypass rate of 92.7% for safety filters.
  arXiv  Detail & Related papers  (2024-12-14T12:11:26Z)
• Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In [5.65782619470663]
  We examine how ReAct agents can be exploited using a straightforward yet effective method we refer to as the foot-in-the-door attack. Our experiments show that indirect prompt injection attacks can significantly increase the likelihood of the agent performing subsequent malicious actions. To mitigate this vulnerability, we propose implementing a simple reflection mechanism that prompts the agent to reassess the safety of its actions during execution.
  arXiv  Detail & Related papers  (2024-10-22T12:24:41Z)
• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents [84.96249955105777]
  LLM agents may pose a greater risk if misused, but their robustness remains underexplored. We propose a new benchmark called AgentHarm to facilitate research on LLM agent misuse. We find leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking.
  arXiv  Detail & Related papers  (2024-10-11T17:39:22Z)
• SecAlign: Defending Against Prompt Injection with Preference Optimization [52.48001255555192]
  Adrial prompts can be injected into external data sources to override the system's intended instruction and execute a malicious instruction.<n>We propose a new defense called SecAlign based on the technique of preference optimization.<n>Our method reduces the success rates of various prompt injections to around 0%, even against attacks much more sophisticated than ones seen during training.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Context [49.13497493053742]
  This research explores converting a nonsensical suffix attack into a sensible prompt via a situation-driven contextual re-writing. We combine an independent, meaningful adversarial insertion and situations derived from movies to check if this can trick an LLM. Our approach demonstrates that a successful situation-driven attack can be executed on both open-source and proprietary LLMs.
  arXiv  Detail & Related papers  (2024-07-19T19:47:26Z)
• AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases [73.04652687616286]
  We propose AgentPoison, the first backdoor attack targeting generic and RAG-based LLM agents by poisoning their long-term memory or RAG knowledge base. Unlike conventional backdoor attacks, AgentPoison requires no additional model training or fine-tuning. On each agent, AgentPoison achieves an average attack success rate higher than 80% with minimal impact on benign performance.
  arXiv  Detail & Related papers  (2024-07-17T17:59:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.