MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents

• URL: http://arxiv.org/abs/2605.03482v2
• Date: Thu, 07 May 2026 11:41:14 GMT
• Title: MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents
• Abstract summary: We formalize memory poisoning attacks on retrieval-augmented agents as a Stackelberg game with a unified evaluation framework.<n>We show faithful evaluation increases measured attack success by $4times$ (ASR-R: 0.25 to 1.00$)<n>Our primary contribution is MEMSAD, a calibration-based defense grounded in a gradient coupling.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Persistent external memory enables LLM agents to maintain context across sessions, yet its security properties remain formally uncharacterized. We formalize memory poisoning attacks on retrieval-augmented agents as a Stackelberg game with a unified evaluation framework spanning three attack classes with escalating access assumptions. Correcting an evaluation protocol inconsistency in the triggered-query specification of Chen et al. (2024), we show faithful evaluation increases measured attack success by $4\times$ (ASR-R: $0.25 \to 1.00$). Our primary contribution is MEMSAD (Semantic Anomaly Detection), a calibration-based defense grounded in a gradient coupling theorem: under encoder regularity, the anomaly score gradient and the retrieval objective gradient are provably identical, so any continuous perturbation that reduces detection risk necessarily degrades retrieval rank. This coupling yields a certified detection radius guaranteeing correct classification regardless of adversary strategy. We prove minimax optimality via Le Cam's method, showing any threshold detector requires $Ω(1/ρ^2)$ calibration samples and MEMSAD achieves this up to $\log(1/δ)$ factors. We further derive online regret bounds for rolling calibration at rate $O(σ^{2/3}Δ^{1/3})$, and formally characterize a discrete synonym-invariance loophole that marks the boundary of what continuous-space defenses can guarantee. Experiments on a $3 \times 5$ attack-defense matrix with bootstrap confidence intervals, Bonferroni-corrected hypothesis tests, and Clopper-Pearson validation ($n=1{,}000$) confirm: composite defenses achieve TPR $= 1.00$, FPR $= 0.00$ across all attacks, while synonym substitution evades detection at $Δ$ ASR-R $\approx 0$, exposing a gap existing embedding-based defenses cannot close.

Related papers

• Memorization Is Not Extraction: Tight Differential-Privacy Bounds and Audit Blind Spots [9.539275654775706]
  Memorization in large language models is measured through a zoo of definitions whose formal relations are unknown.<n>We show that differential privacy (DP) is treated as a proxy against all of them at once.
  arXiv  Detail & Related papers  (2026-08-27T23:28:08Z)
• Repeated-Game Security for Restaking-Based Verifiable Inference [2.9661360665913574]
  This paper shows that a one-round slashing condition can overstate security when inference is supplied repeatedly under the same stake.<n>We propose a deployable mechanism combining history-dependent challenges, reputation-weighted slashing, and stake vesting.
  arXiv  Detail & Related papers  (2026-08-10T03:03:56Z)
• Detecting and Discriminating Operator Misspecification in Hybrid PDE-Parameter Learning: a Reference-Free Instrument, with Discrimination Bounded In Sample [0.0]
  We build an instrument that reads from a single fit and with no oracle.<n>We report where the instrument is blind, a pre-registered negative where a neural estimator loses to Tikhonov-regularized inversion at recovery.
  arXiv  Detail & Related papers  (2026-08-04T21:20:44Z)
• Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs [4.721990925113432]
  Aligned language models routinely misreport under non-evidential pressure.<n>We study the two demands, resist (ignore forbidden pressure) and update (follow licensed evidence)<n>We causally localize low-rank report coordinates for answer, confidence, and caveat.
  arXiv  Detail & Related papers  (2026-07-14T17:28:25Z)
• Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols [51.56484100374058]
  We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
  arXiv  Detail & Related papers  (2026-04-20T13:25:40Z)
• EdgeDetect: Importance-Aware Gradient Compression with Homomorphic Aggregation for Federated Intrusion Detection [0.0]
  Federated learning (FL) enables collaborative intrusion detection without raw data exchange.<n> conventional FL incurs high communication overhead from full-precision transmission.<n>This paper presents EdgeDetect, a privacy-aware federated IDS for bandwidth-constrained 6G-IoT environments.
  arXiv  Detail & Related papers  (2026-04-16T06:16:14Z)
• VeriX-Anon: A Multi-Layered Framework for Mathematically Verifiable Outsourced Target-Driven Data Anonymization [0.0]
  VeriX-Anon is a multi-layered verification framework for outsourced Target-Driven k-anonymization.<n>It combines deterministic verification via Merkle-style hashing of an Authenticated Decision Tree, probabilistic verification via Boundary Sentinels near the Random Forest decision boundary and exact-duplicate Twins with cryptographic identifiers.<n>It correctly detected deviations in 11 of 12 scenarios.
  arXiv  Detail & Related papers  (2026-04-14T08:22:18Z)
• Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropout [1.3902537392439644]
  We study distributed optimization over networks with Byzantine agents that may send arbitrary adversarial messages.<n>We propose emphGradient Tracking with Probabilistic Edge Dropout and Leaky Integration (GT-PD-L)<n>GT-PD-L outperforms coordinate-wise mean by up to 4.3 percentage points under stealth attacks.
  arXiv  Detail & Related papers  (2026-04-01T03:55:42Z)
• Robustness, Cost, and Attack-Surface Concentration in Phishing Detection [0.0]
  Phishing detectors built on engineered website features attain near-perfect accuracy under i.i.d. evaluation.<n>We study this gap through a cost-aware evasion framework that models discrete, monotone feature edits under explicit attacker budgets.
  arXiv  Detail & Related papers  (2026-03-19T17:53:32Z)
• Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models [68.45272703833209]
  We show that state-of-the-art PRMs are systematically exploitable under adversarial optimization pressure.<n>We introduce a three-tiered diagnostic framework that applies increasing adversarial pressure to quantify these vulnerabilities.<n>We release PRM-BiasBench and a diagnostic toolkit to enable robustness evaluation before deployment.
  arXiv  Detail & Related papers  (2026-02-20T23:38:03Z)
• Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning [61.07540493350384]
  Self-distillation (SD) is the process of retraining a student on a mixture of ground-truth and the teacher's own predictions.<n>We show that for any prediction risk, the optimally mixed student improves upon the ridge teacher for every regularization level.<n>We propose a consistent one-shot tuning method to estimate $star$ without grid search, sample splitting, or refitting.
  arXiv  Detail & Related papers  (2026-02-19T17:21:15Z)
• Spectral Sentinel: Scalable Byzantine-Robust Decentralized Federated Learning via Sketched Random Matrix Theory on Blockchain [0.0]
  Byzantine clients poison gradients under heterogeneous (Non-IID) data.<n>We propose Spectral Sentinel, a Byzantine detection and aggregation framework.<n>We implement the full system with blockchain integration on Polygon networks.
  arXiv  Detail & Related papers  (2025-12-14T09:43:03Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification [63.65630243675792]
  Diffusion-based purification defenses leverage diffusion models to remove crafted perturbations of adversarial examples. Recent studies show that even advanced attacks cannot break such defenses effectively. We propose a unified framework DiffAttack to perform effective and efficient attacks against diffusion-based purification defenses.
  arXiv  Detail & Related papers  (2023-10-27T15:17:50Z)
• Detection as Regression: Certified Object Detection by Median Smoothing [50.89591634725045]
  This work is motivated by recent progress on certified classification by randomized smoothing. We obtain the first model-agnostic, training-free, and certified defense for object detection against $ell$-bounded attacks.
  arXiv  Detail & Related papers  (2020-07-07T18:40:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.