Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

• URL: http://arxiv.org/abs/2607.18063v1
• Date: Mon, 20 Jul 2026 15:30:38 GMT
• Title: Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
• Abstract summary: LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation.<n>We present a 21-scenario benchmark for emphadaptive multi-round attacks against memoryless LLM defenders.
• Score: 2.3029886838540032
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round attacks against memoryless LLM defenders}: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. Holding the 21 scenarios, attackers, defenders, and structured-output scoring fixed, restricting scoring to the first attacker turn yields $0$-$1\%$ attack success rate (ASR); allowing 15 rounds of adaptive attack yields $5.4$-$14.0\%$. Pooling three frontier attacker LLMs uncovers $1.4$-$2.2\times$ as many unique successful attacks as the best single attacker, and the generated attacks have low cosine similarity ($0.02$-$0.14$) to attacks in existing benchmarks. Claude Opus 4.6 and GPT-5.4 are tied in aggregate ($5.4\%$ each; overlapping $95\%$ CIs), but their weaknesses differ sharply: on one scenario Opus reaches $60\%$ ASR ($95\%$ CI $36$--$80\%$) while GPT-5.4 and Gemini each stay at $7\%$ (CI $1$-$30\%$; the gap is preserved in a higher-$N$ replication). $13$ of $21$ scenarios distinguish at least one defender pair, yet rankings disagree across scenarios (Kendall's $W = 0.19$). We release the benchmark -- 21 evaluation scenarios, 10 public development scenarios, the orchestrator, baseline harnesses, and a multi-attacker CLI -- plus 945 transcripts from the 3$\times$3 frontier matrix, an attack-replay dataset, and 18{,}422 gpt-oss-20b battles from an open competition's final scoring rounds.

Related papers

• Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models [25.860894697929737]
  We propose Meta-Adaptive Multimodal Jailbreaking (MAMJ) to optimize the attacker itself along two axes: an attack strategy prompt (ASP) governing attack iteration and attacker model weights determining attack effectiveness.<n>On MM-SafetyBench, MAMJ achieves 81.0%, 78.9%, and 82.3% ASR against GPT-4o, Gemini-3-Pro-adaptive, and Seed 2.0, respectively, outperforming the strongest sample-level baseline by up to 24.1 percentage points.
  arXiv  Detail & Related papers  (2026-08-27T15:23:22Z)
• Safety, or Just Capability? A Validity Audit of Agent-Safety Benchmarks [10.839956062772792]
  Three agent-safety benchmarks rank the same 18 models differently.<n>R-Judge specificity against AgentHarm safety correlates $-0.64$ at $n=7$ and $+0.02$ at $n=18$, and a quarter of random size-7 subsets reach $|| geq 0.5$ around that near-zero value.<n>AgentHarm shows the strongest held-out association, $=+0.72$ with three-template jailbreak safety after controlling capability.
  arXiv  Detail & Related papers  (2026-07-30T03:45:10Z)
• A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG [4.879573049082894]
  We release a benchmark and failure-mode-aware evaluation framework for grounded QA under coordinated retrieval poisoning.<n>The framework partitions reader outputs into four mutually exclusive categories (emphgold, emphhijack, emphabstention, emphdrift)<n>We introduce emphpolymorphic sybil poisoning, a coordinated attack class in which $S$ lexically diverse passages jointly support an attacker-chosen target.
  arXiv  Detail & Related papers  (2026-07-04T06:56:06Z)
• BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents [10.767742708000897]
  Stepwise group-based RL is an attractive way to train long-horizon LLM agents without a learned critic.<n>We show that current agentic variants violate this assumption through a state-action credit mismatch.<n>We introduce BiPACE, a drop-in advantage estimator that fixes both sides without adding a critic, auxiliary loss, or extra rollouts.
  arXiv  Detail & Related papers  (2026-06-24T08:32:42Z)
• Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing [51.56484100374058]
  Production LLM applications stack several defense families -- refusal-phrase filters, tokenbudget controls, model allowlists, rate limits, tool-registry authentication -- yetBAS benchmarks report a single aggregate coverage number.<n>We add four-LLM-Top-10-aware agents to a 21-agent baseline scanner and target a lattice of four synthetic LLM endpoints.
  arXiv  Detail & Related papers  (2026-06-01T19:39:25Z)
• Hidden-State Privacy Has an Empty Middle [51.56484100374058]
  We prove a complementary Fisher-ball lower bound: every full-rank Gaussian release at $O(1)$ Fisher utility admits a direction whose Mahalanobis signal grows linearly in hidden width.<n>A split-memory transformer trained from scratch reaches $G_mathrmMah in [20, 33]$ at 90M and maintains a $6$--$24times$ advantage over same-budget GPT baselines from 30M to 1B at a fixed-token language loss penalty.
  arXiv  Detail & Related papers  (2026-05-21T20:12:09Z)
• Fast Proxies for LLM Robustness Evaluation [48.53873823665833]
  We compare the ability of fast proxy metrics to predict the real-world robustness of an LLM against a simulated attacker ensemble.<n>This allows us to estimate a model's robustness to computationally expensive attacks without requiring runs of the attacks themselves.
  arXiv  Detail & Related papers  (2025-02-14T11:15:27Z)
• Deep Adversarial Defense Against Multilevel-Lp Attacks [5.604868766260297]
  This paper introduces a computationally efficient multilevel $ell_p$ defense, called the Efficient Robust Mode Connectivity (EMRC) method. Similar to analytical continuation approaches used in continuous optimization, the method blends two $p$-specific adversarially optimal models. We present experiments demonstrating that our approach performs better on various attacks as compared to AT-$ell_infty$, E-AT, and MSD.
  arXiv  Detail & Related papers  (2024-07-12T13:30:00Z)
• MultiRobustBench: Benchmarking Robustness Against Multiple Attacks [86.70417016955459]
  We present the first unified framework for considering multiple attacks against machine learning (ML) models. Our framework is able to model different levels of learner's knowledge about the test-time adversary. We evaluate the performance of 16 defended models for robustness against a set of 9 different attack types.
  arXiv  Detail & Related papers  (2023-02-21T20:26:39Z)
• Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attacks [25.053383672515697]
  We propose a novel defense, namely Adversarial Attack on Attackers (AAA), to confound SQAs towards incorrect attack directions. In this way, SQAs are prevented regardless of the model's worst-case robustness.
  arXiv  Detail & Related papers  (2022-05-24T15:10:50Z)
• Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack [96.50202709922698]
  A practical evaluation method should be convenient (i.e., parameter-free), efficient (i.e., fewer iterations) and reliable. We propose a parameter-free Adaptive Auto Attack (A$3$) evaluation method which addresses the efficiency and reliability in a test-time-training fashion.
  arXiv  Detail & Related papers  (2022-03-10T04:53:54Z)
• Composite Adversarial Attacks [57.293211764569996]
  Adversarial attack is a technique for deceiving Machine Learning (ML) models. In this paper, a new procedure called Composite Adrial Attack (CAA) is proposed for automatically searching the best combination of attack algorithms. CAA beats 10 top attackers on 11 diverse defenses with less elapsed time.
  arXiv  Detail & Related papers  (2020-12-10T03:21:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.