Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful LLM agents

• URL: http://arxiv.org/abs/2605.08442v1
• Date: Fri, 08 May 2026 20:04:57 GMT
• Title: Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful LLM agents
• Abstract summary: Persistent memory attacks against LLM agents achieve high attack success rates against open-source models.<n>In these attacks, malicious instructions injected via RAG-retrieved documents are stored in persistent memory and executed in later sessions.<n>We evaluate six defenses across four architectural layers against delayed-trigger attacks on nine open-source models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Persistent memory attacks against LLM agents achieve high attack success rates against open-source models. In these attacks, malicious instructions injected via RAG-retrieved documents are stored in persistent memory and executed in later sessions. However, no systematic evaluation of defense effectiveness against this attack class exists. We evaluate six defenses across four architectural layers against delayed-trigger attacks on nine open-source models (5,040 runs, N=40 per condition). Four defenses fail at approximately baseline attack success rate: input-level filtering (Minimizer, Sanitizer) and retrieval-level filtering (RAG Sanitizer, RAG LLM Judge) achieve 88-89% ASR, statistically indistinguishable from the undefended baseline of 88.6%. Prompt Hardening partially fails at 77.8% ASR, with the reduction driven by two models at 0%: one genuine defense effect and one model-level refusal independent of the defense. The architectural explanation holds: input-level defenses cannot observe RAG-injected content, and retrieval-level classifiers are defeated by compliance-framed semantic masking. One defense, tool-gating at the memory layer (Memory Sandbox), reduces ASR to 0% for eight of nine models by removing the recall capability the attack requires. The exception inverts the defense entirely: a reasoning model that achieves 0% ASR under no defense via execution refusal inverts to 100% ASR under Memory Sandbox, because removing explicit recall forces the model onto the RAG pathway where its refusal mechanism does not activate. Memory Sandbox imposes zero utility cost in the absence of attack (BTCR = 100% across all conditions). These results provide the first systematic characterization of why each defense class fails against persistent memory attacks, enabling informed defense investment decisions.

Related papers

• Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration [37.700303596218184]
  Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA)<n>We introduce Decoy Direction Optimization (DDO), a fast, post-hoc weight-editing defense that requires no base-model finetuning.<n>We prove a spectral bound formalizing this effect and evaluate DDO across six model families, achieving 10% ASR under standard RFA.
  arXiv  Detail & Related papers  (2026-09-14T18:36:35Z)
• Low-ASR Backdoors: Exploiting Attack Success Rate Reduction and Attacker-Defender Asymmetry [1.720105558875703]
  We introduce a reverse-training framework that weakens the trigger-target association, producing low-ASR backdoor models.<n>We show that state-of-the-art defenses fail consistently under low-ASR conditions.
  arXiv  Detail & Related papers  (2026-08-27T15:56:44Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• Quantization Blindspots: How Model Compression Breaks Backdoor Defenses [1.8646269335840449]
  We study how backdoor defenses behave under standard quantization pipelines.<n>We find that INT8 quantization reduces the detection rate of all evaluated defenses to 0% while leaving attack success rates above 99%.<n>Our results expose a mismatch between how defenses are commonly evaluated and how models are actually deployed.
  arXiv  Detail & Related papers  (2025-12-06T02:04:32Z)
• Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systems [11.812488957698038]
  Large language models (LLMs) are reshaping numerous facets of our daily lives, leading widespread adoption as web-based services.<n>Retrieval-Augmented Generation (RAG) has emerged as a promising direction by generating responses grounded in external knowledge sources.<n>Recent studies demonstrate the vulnerability of RAG, such as knowledge corruption attacks by injecting misleading information.<n>In this work, we introduce RAGDefender, a resource-efficient defense mechanism against knowledge corruption.
  arXiv  Detail & Related papers  (2025-11-03T06:39:58Z)
• The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections [74.60337113759313]
  Current defenses against jailbreaks and prompt injections are typically evaluated against a static set of harmful attack strings.<n>We argue that this evaluation process is flawed. Instead, we should evaluate defenses against adaptive attackers who explicitly modify their attack strategy to counter a defense's design.
  arXiv  Detail & Related papers  (2025-10-10T05:51:04Z)
• Data Free Backdoor Attacks [83.10379074100453]
  DFBA is a retraining-free and data-free backdoor attack without changing the model architecture.<n>We verify that our injected backdoor is provably undetectable and unchosen by various state-of-the-art defenses.<n>Our evaluation on multiple datasets demonstrates that our injected backdoor: 1) incurs negligible classification loss, 2) achieves 100% attack success rates, and 3) bypasses six existing state-of-the-art defenses.
  arXiv  Detail & Related papers  (2024-12-09T05:30:25Z)
• Evaluating the Robustness of the "Ensemble Everything Everywhere" Defense [90.7494670101357]
  Ensemble everything everywhere is a defense to adversarial examples.<n>We show that this defense is not robust to adversarial attack.<n>We then use standard adaptive attack techniques to reduce the defense's robust accuracy.
  arXiv  Detail & Related papers  (2024-11-22T10:17:32Z)
• Protecting against simultaneous data poisoning attacks [14.893813906644153]
  Current backdoor defense methods are evaluated against a single attack at a time. We show that simultaneously executed data poisoning attacks can effectively install multiple backdoors in a single model. We develop a new defense, BaDLoss, that is effective in the multi-attack setting.
  arXiv  Detail & Related papers  (2024-08-23T16:57:27Z)
• Hijacking Attacks against Neural Networks by Analyzing Training Data [21.277867143827812]
  CleanSheet is a new model hijacking attack that obtains the high performance of backdoor attacks without requiring the adversary to train the model. CleanSheet exploits vulnerabilities in tampers stemming from the training data. Results show that CleanSheet exhibits comparable to state-of-the-art backdoor attacks, achieving an average attack success rate (ASR) of 97.5% on CIFAR-100 and 92.4% on GTSRB.
  arXiv  Detail & Related papers  (2024-01-18T05:48:56Z)
• Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning [71.17774313301753]
  We explore the robustness of self-supervised learned high-level representations by using them in the defense against adversarial attacks. Experimental results on the ASVspoof 2019 dataset demonstrate that high-level representations extracted by Mockingjay can prevent the transferability of adversarial examples.
  arXiv  Detail & Related papers  (2020-06-05T03:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.