Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

• URL: http://arxiv.org/abs/2608.05108v1
• Date: Wed, 05 Aug 2026 17:44:09 GMT
• Title: Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming
• Abstract summary: We develop PIMiner, an agentic system for prompt injection red-teaming.<n>During training, PIMiner is trained on a sequence of (dataset, target model) pairs.<n>At test time, the learned strategy library can be directly transferred to a previously unseen target LLM.
• Score: 19.85522880335499
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting training data to improve defenses. Existing state-of-the-art prompt injection red-teaming methods primarily rely on reinforcement learning (RL), producing attacker models that often generalize poorly to new target LLMs. In this work, we develop PIMiner, an agentic system for prompt injection red-teaming. During training, PIMiner is trained on a sequence of (dataset, target model) pairs and builds a strategy library from scratch. At test time, the learned strategy library can be directly transferred to a previously unseen target LLM without additional training. PIMiner requires only a small number of queries to a target agent (e.g., 10) per test sample. Experimental results demonstrate that PIMiner achieves strong performance. On IPIArena, it attains a 76.2% ASR against Gemini-2.5-Pro, 61.9% ASR against GPT-5.1, and 42.9% ASR against Claude-Sonnet-4.5. On AgentDojo, it achieves an 86.7% ASR against Gemini-2.5-Pro, 53.3% ASR against GPT-5.1, and 40.0% ASR against Claude-Sonnet-4.5.

Related papers

• Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction [62.42945715963878]
  Repository-level vulnerability reproduction is a demanding software engineering (SE) task.<n>Recent LLM agents can often execute these steps when the approach is correct, yet they still fail by choosing the wrong strategy.<n>This paper argues that strategy, rather than the full action trajectory, is the right learning unit for such SE agents.
  arXiv  Detail & Related papers  (2026-07-02T06:27:27Z)
• Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare [0.0]
  This paper applies the AI Trust, Risk, and Security Management (TRiSM) framework to a medical report-generation application.<n>It demonstrates how an insecure agent workflow can be transformed into security-conscious agentic workflow.
  arXiv  Detail & Related papers  (2026-06-27T01:00:39Z)
• Agentic Knowledge Distillation: Autonomous Training of Small Language Models for SMS Threat Detection [0.4899818550820574]
  Agentic Knowledge Distillation consists of a powerful LLM that fine-tunes a smaller student SLM, deployable for security tasks without human intervention.<n>We compare four LLMs in this teacher role on SMS spam/smishing detection with two student SLMs.<n>Our results show that performance varies substantially depending on the teacher LLM, with the best configuration achieving 94.31% accuracy and 96.25% recall.
  arXiv  Detail & Related papers  (2026-02-11T13:57:56Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning [18.219912912964812]
  We present SIRAJ: a generic red-teaming framework for arbitrary black-box LLM agents.<n>We employ a dynamic two-step process that starts with an agent definition and generates diverse seed test cases.<n>It iteratively constructs and refines model-based adversarial attacks based on the execution trajectories of former attempts.
  arXiv  Detail & Related papers  (2025-10-30T00:32:58Z)
• Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People [81.63702981397408]
  Given limited resources, to what extent do agents based on language models (LMs) act rationally?<n>We develop methods to benchmark and enhance agentic information-seeking, drawing on insights from human behavior.<n>For Spotter agents, our approach boosts accuracy by up to 14.7% absolute over LM-only baselines; for Captain agents, it raises expected information gain (EIG) by up to 0.227 bits (94.2% of the achievable noise ceiling)
  arXiv  Detail & Related papers  (2025-10-23T17:57:28Z)
• Adversarial Reinforcement Learning for Large Language Model Agent Safety [20.704989548285372]
  Large Language Model (LLM) agents can leverage tools like Google Search to complete complex tasks.<n>Current defense strategies rely on fine-tuning LLM agents on datasets of known attacks.<n>We propose Adversarial Reinforcement Learning for Agent Safety (ARLAS), a novel framework that leverages adversarial reinforcement learning (RL) by formulating the problem as a two-player zero-sum game.
  arXiv  Detail & Related papers  (2025-10-06T23:09:18Z)
• AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? [71.21547572568655]
  AgenTracer-8B is a lightweight failure tracer trained with multi-granular reinforcement learning.<n>On the Who&When benchmark, AgenTracer-8B outperforms giant proprietary LLMs like Gemini-2.5-Pro and Claude-4-Sonnet by up to 18.18%.<n>AgenTracer-8B delivers actionable feedback to off-the-shelf multi-agent systems like MetaGPT and MaAS with 4.8-14.2% performance gains.
  arXiv  Detail & Related papers  (2025-09-03T13:42:14Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy [47.42940885853956]
  A$3$T is a framework that enables the Autonomous. of Agent Trajectories in the style of ReAct. In AlfWorld, the agent trained with A$3$T obtains a 1-shot success rate of 96%, and 100% success with 4 iterative rounds.
  arXiv  Detail & Related papers  (2024-03-21T17:43:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.