AgentDrift: Unsafe Recommendation Drift Under Tool Corruption Hidden by Ranking Metrics in LLM Agents

• URL: http://arxiv.org/abs/2603.12564v2
• Date: Wed, 18 Mar 2026 20:31:03 GMT
• Title: AgentDrift: Unsafe Recommendation Drift Under Tool Corruption Hidden by Ranking Metrics in LLM Agents
• Abstract summary: We introduce a paired-trajectory protocol that replays real financial dialogues under clean and contaminated tool-output conditions.<n>We observe the evaluation-blindness pattern: recommendation quality is largely preserved under contamination.<n>Even narrative-only corruption (biased headlines, no numerical manipulation) induces significant drift while completely evading consistency monitors.
• Score: 2.995458991057093
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tool-augmented LLM agents increasingly serve as multi-turn advisors in high-stakes domains, yet their evaluation relies on ranking-quality metrics that measure what is recommended but not whether it is safe for the user. We introduce a paired-trajectory protocol that replays real financial dialogues under clean and contaminated tool-output conditions across seven LLMs (7B to frontier) and decomposes divergence into information-channel and memory-channel mechanisms. Across the seven models tested, we consistently observe the evaluation-blindness pattern: recommendation quality is largely preserved under contamination (utility preservation ratio approximately 1.0) while risk-inappropriate products appear in 65-93% of turns, a systematic safety failure poorly reflected by standard NDCG. Safety violations are predominantly information-channel-driven, emerge at the first contaminated turn, and persist without self-correction over 23-step trajectories; no agent across 1,563 contaminated turns explicitly questions tool-data reliability. Even narrative-only corruption (biased headlines, no numerical manipulation) induces significant drift while completely evading consistency monitors. A safety-penalized NDCG variant (sNDCG) reduces preservation ratios to 0.51-0.74, indicating that much of the evaluation gap becomes visible once safety is explicitly measured. These results motivate considering trajectory-level safety monitoring, beyond single-turn quality, for deployed multi-turn agents in high-stakes settings.

Related papers

• BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents [20.217145643468598]
  We introduce Blindspot, a benchmark for trajectory-level safety calibration of long-horizon tool-using agents.<n>Blindspot evaluates complete user-agent-environment trajectories through adaptive adversarial interaction, stateful tool execution, and execution-grounded adjudication.<n>We evaluate 13 proprietary and open-weight LLMs using eight metrics covering unsafe completion, appropriate refusal, benign utility, over-refusal, repeated-run robustness, and post-refusal failure.
  arXiv  Detail & Related papers  (2026-09-14T20:12:27Z)
• From Detection to Refusal: Safer LLMs via Circuit-Guided Weight Scaling [23.223636694641502]
  Large Language Models (LLMs) are vulnerable to generating unsafe content under adversarial prompting.<n>We study safety from a mechanistic interpretability perspective and characterize a multi-stage *safety circuit*<n>We show that suppressing upstream Harmful Detection Heads disrupts downstream refusal behavior and that safety neurons mediate this interaction.
  arXiv  Detail & Related papers  (2026-08-30T08:01:42Z)
• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs [20.59321114618083]
  We show that single-turn diagnostics systematically overestimate RAG safety, that contradiction is uncorrelated with safe resolution, and that no universal prompt fix exists.<n>The gap between what models recognize and what they do must be measured and closed before retrieval-augmented systems can be trusted in high-stakes settings.
  arXiv  Detail & Related papers  (2026-05-26T15:18:43Z)
• On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment [54.30690671490447]
  Existing safety-alignment signals are largely response-level or off-policy.<n>FATE transforms verifier-scored failures into repair supervision without expert demonstrations.<n>FATE reduces attack success rate by 33.5%, harmful compliance by 82.6%, and improves external trajectory-safety diagnosis by 6.5%.
  arXiv  Detail & Related papers  (2026-05-12T09:56:28Z)
• OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences [64.01706941950489]
  Current safety paradigms primarily target malicious intent or situational violations.<n>We propose shifting the safety frontier toward consequence-driven safety, a paradigm essential for the robust deployment of autonomous and embodied agents.<n>We develop the Consequence-Aware Safety Policy Optimization (CASPO) framework, which integrates the model's intrinsic reasoning as a dynamic reference for token-level self-distillation rewards.
  arXiv  Detail & Related papers  (2026-03-10T14:16:43Z)
• Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation [2.102846336724103]
  Procedure-Aware Evaluation (PAE) is a framework that formalizes agent procedures as structured observations.<n>We evaluate state-of-the-art Large Language Model (LLM)-based agents on tau-bench.
  arXiv  Detail & Related papers  (2026-03-03T15:47:41Z)
• Safety-Guided Flow (SGF): A Unified Framework for Negative Guidance in Safe Generation [2.5489046505746704]
  In this paper, we introduce a unified probabilistic framework using a Maximum Mean Discrepancy (MMD) potential for image generation tasks.<n>We leverage control-barrier functions analysis to justify the existence of a critical time window in which negative guidance must be strong.<n>We evaluate our unified framework on several realistic safe generation scenarios, confirming that negative guidance should be applied in the early stages of the denoising process.
  arXiv  Detail & Related papers  (2026-03-02T07:31:29Z)
• ThreatFormer-IDS: Robust Transformer Intrusion Detection with Zero-Day Generalization and Explainable Attribution [0.0]
  Intrusion detection in IoT and industrial networks requires models that can detect rare attacks at low false-positive rates while remaining reliable under evolving traffic and limited labels.<n>We propose ThreatFormer- IDS, a Transformer-based sequence modeling framework that converts flow records into time-ordered windows and learns contextual representations for robust intrusion screening.<n>On the ToN IoT benchmark with chronological evaluation, ThreatFormer-IDS achieves AUCROC 0.994, AUC-PR 0.956, and Recall@1%FPR 0.910, outperforming strong tree-based and sequence baselines.
  arXiv  Detail & Related papers  (2026-02-26T23:20:42Z)
• DRAFT: Task Decoupled Latent Reasoning for Agent Safety [59.46137757545185]
  We propose DRAFT (Task Decoupled Latent Reasoning for Agent Safety), a latent reasoning framework that decouples safety judgment into two trainable stages.<n>An Extractor distills the full trajectory into a compact continuous latent draft, and a Reasoner that jointly attends to the draft and the original trajectory to predict safety.<n>DRAFT consistently outperforms strong baselines, improving accuracy from 63.27% (LoRA) to 91.18% averaged over benchmarks.
  arXiv  Detail & Related papers  (2026-02-11T07:45:14Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• GuardEval: A Multi-Perspective Benchmark for Evaluating Safety, Fairness, and Robustness in LLM Moderators [9.212268642636007]
  We present GuardEval, a benchmark dataset for training and evaluation of large language models (LLMs)<n>We also present GemmaGuard (GGuard), a fine-tuned version of Gemma3-12B trained on GuardEval, to assess content moderation with fine-grained labels.<n>We show that multi-perspective, human-centered safety benchmarks are critical for reducing biased and inconsistent moderation decisions.
  arXiv  Detail & Related papers  (2025-12-22T14:49:28Z)
• DUAL-Bench: Measuring Over-Refusal and Robustness in Vision-Language Models [59.45605332033458]
  Safety mechanisms can backfire, causing over-refusal, where models decline benign requests out of excessive caution.<n>No existing benchmark has systematically addressed over-refusal in the visual modality.<n>This setting introduces unique challenges, such as dual-use cases where an instruction is harmless, but the accompanying image contains harmful content.
  arXiv  Detail & Related papers  (2025-10-12T23:21:34Z)
• Preemptive Detection and Steering of LLM Misalignment via Latent Reachability [8.01833277608166]
  Large language models (LLMs) are now ubiquitous in everyday tools, raising urgent safety concerns about their tendency to generate harmful content.<n>We propose BRT-Align, a reachability-based framework that brings control-theoretic safety tools to LLM inference.
  arXiv  Detail & Related papers  (2025-09-25T20:15:29Z)
• Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency [17.57889200051214]
  Fine-tuning a general-purpose large language model (LLM) for a specific domain or task has become a routine procedure for ordinary users.<n>We consider this to be a critical failure mode of LLMs due to the widespread uptake of fine-tuning, combined with the benign nature of the "attack"<n>Our experiments expose surprising variance in the results of the safety evaluation, even when seemingly inconsequential changes are made to the fine-tuning setup.
  arXiv  Detail & Related papers  (2025-06-20T17:57:12Z)
• Agent-SafetyBench: Evaluating the Safety of LLM Agents [72.92604341646691]
  We introduce Agent-SafetyBench, a benchmark designed to evaluate the safety of large language models (LLMs)<n>Agent-SafetyBench encompasses 349 interaction environments and 2,000 test cases, evaluating 8 categories of safety risks and covering 10 common failure modes frequently encountered in unsafe interactions.<n>Our evaluation of 16 popular LLM agents reveals a concerning result: none of the agents achieves a safety score above 60%.
  arXiv  Detail & Related papers  (2024-12-19T02:35:15Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users.<n>We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions.<n>We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.