Rethinking Indirect Prompt Injection as a Test-Time Search Problem

• URL: http://arxiv.org/abs/2609.04495v1
• Date: Thu, 03 Sep 2026 21:28:37 GMT
• Title: Rethinking Indirect Prompt Injection as a Test-Time Search Problem
• Abstract summary: We formulate indirect prompt injection as a test-time search over a task-dependent attack surface.<n>We introduce an agentic attacker with a dedicated search harness that performs environment reconnaissance.<n>We find that increasing attacker test-time compute improves vulnerability discovery and exploitation.
• Score: 3.247379743630575
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We formulate indirect prompt injection as a test-time search over a task-dependent attack surface induced by the environment, user task, and injection task. To operationalize this formulation, we introduce an agentic attacker with a dedicated search harness that performs environment reconnaissance, structured reasoning over attack strategies, and adaptive evaluation using victim-agent feedback. Across heterogeneous tasks, we find that increasing attacker test-time compute improves vulnerability discovery and exploitation, while ablations show that explicit strategy management is important for avoiding redundant search and sustaining gains at larger budgets. These results suggest that agentic security evaluations should characterize both the attacker's search procedure and compute budget, rather than treating attack success as a budget-independent property of the victim. More broadly, our findings identify the attacker's adaptive search over the system attack surfaces as an important and underexplored security risk for tool-using agents.

Related papers

• Breadcrumbing Search Agents [118.20284823674382]
  LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk.<n>We show that the channel delivering search and page observations is a fragile security boundary.<n>Under a constrained tool-intermediary threat model, appending only one controlled result per query can substantially increase attack success.
  arXiv  Detail & Related papers  (2026-08-05T07:57:27Z)
• PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections [51.8246149916068]
  Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments.<n>We propose PI-Hunter, an automated agentic auditing framework for proactive vulnerability exposure in LLM agents.
  arXiv  Detail & Related papers  (2026-06-10T22:57:02Z)
• WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents [53.84430233754606]
  We introduce WMAttack, an automated attack-search framework for adversarial evaluation of world-model agents.<n> WMAttack formulates robustness evaluation as a finite-budget search over attack configurations.<n>We show that WMAttack consistently discovers stronger attacks than the evaluated baselines.
  arXiv  Detail & Related papers  (2026-05-22T04:31:09Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems [101.68501850486179]
  We explore adversarial attacks against retrieval-augmented generation (RAG) systems to identify their vulnerabilities.<n>This task aims to find imperceptible perturbations that retrieve a target document, originally excluded from the initial top-$k$ candidate set.<n>We propose ReGENT, a reinforcement learning-based framework that tracks interactions between the attacker and the target RAG.
  arXiv  Detail & Related papers  (2025-05-24T08:19:25Z)
• Stealthy Multi-Task Adversarial Attacks [17.24457318044218]
  We investigate selectively targeting one task while preserving performance in others within a multi-task framework.<n>This approach is motivated by varying security priorities among tasks in real-world applications, such as autonomous driving.<n>We propose a method for the stealthy multi-task attack framework that utilizes multiple algorithms to inject imperceptible noise into the input.
  arXiv  Detail & Related papers  (2024-11-26T23:18:32Z)
• Fortify the Guardian, Not the Treasure: Resilient Adversarial Detectors [0.0]
  An adaptive attack is one where the attacker is aware of the defenses and adapts their strategy accordingly. Our proposed method leverages adversarial training to reinforce the ability to detect attacks, without compromising clean accuracy. Experimental evaluations on the CIFAR-10 and SVHN datasets demonstrate that our proposed algorithm significantly improves a detector's ability to accurately identify adaptive adversarial attacks.
  arXiv  Detail & Related papers  (2024-04-18T12:13:09Z)
• Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection [12.244543468021938]
  This paper introduces two types of detection tasks for adversarial documents. A benchmark dataset is established to facilitate the investigation of adversarial ranking defense. A comprehensive investigation of the performance of several detection baselines is conducted.
  arXiv  Detail & Related papers  (2023-07-31T16:31:24Z)
• Probabilistic Categorical Adversarial Attack & Adversarial Training [45.458028977108256]
  The existence of adversarial examples brings huge concern for people to apply Deep Neural Networks (DNNs) in safety-critical tasks. How to generate adversarial examples with categorical data is an important problem but lack of extensive exploration. We propose Probabilistic Categorical Adversarial Attack (PCAA), which transfers the discrete optimization problem to a continuous problem that can be solved efficiently by Projected Gradient Descent.
  arXiv  Detail & Related papers  (2022-10-17T19:04:16Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approach [3.5071575478443435]
  Penetration testing plays a critical role in evaluating the security of a target network by emulating real active adversaries. Deep Reinforcement Learning is seen as a promising solution to automating the process of penetration tests. We propose a Chebyshev decomposition critic to find diverse adversary strategies that balance different objectives in the penetration test.
  arXiv  Detail & Related papers  (2022-02-22T02:34:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.