PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis

• URL: http://arxiv.org/abs/2607.12624v1
• Date: Tue, 14 Jul 2026 11:01:10 GMT
• Title: PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis
• Abstract summary: Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks.<n>These agents are expected to comply with not only generic safety guardrails but also purpose-specific restrictions tailored to their designated roles.<n>We propose PVDetector, a training-free framework that detects PI attacks during LLM inference by measuring hidden-state alignment with PV concepts.
• Score: 17.3285594480806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but also purpose-specific restrictions tailored to their designated roles. Such additional restrictions enlarge the attack surface, particularly to prompt injection (PI) attacks. To defend against such attacks, existing detection methods primarily rely on analyzing input-output patterns, yet yield limited effectiveness. To address this limitation, we turn to analyzing the hidden activation space and discover that LLMs inherently retain latent policy-violation (PV) concepts when prompted with requests beyond their designated purpose. Particularly, PV concepts capture the semantics of conflicts between user queries and predefined restrictions, implicitly reflecting LLMs' intrinsic awareness of recognizing policy violations. Building on this insight, we propose PVDetector, a training-free framework that detects PI attacks during LLM inference by measuring hidden-state alignment with PV concepts, which are derived offline from the contrastive pairs of policy-violating and policy-compliant prompts. Experiments across multiple LLMs and datasets show that PVDetector achieves <1\% false negative rate with minimal auxiliary overhead, consistently outperforming state-of-the-art methods. Our code is available at https://github.com/Claresigle/PVDetector .

Related papers

• Steering LLM Viewpoints through Fabricated Evidence Injection [61.22850755490993]
  We introduce a two-phase attack framework that first repackages misleading statements with fabricated rationales, then instruct target LLMs to incorporate these viewpoints when responding to relevant queries.<n>Experiments on BBQ, ToxiGen, and our specialized dataset reveal that commercial LLMs without external safety classifiers remain highly vulnerable.<n>Building on this, we explore multiple defense strategies, among which a tailored safety policy enablesgpt-oss-safeguard to achieve 81% detection rate.
  arXiv  Detail & Related papers  (2026-06-04T14:48:23Z)
• Detector-Evasive LLM Paraphrasing via Constrained Policy Optimization [27.078441680572254]
  AI-text detectors are vulnerable to paraphrasing and detector-guided paraphrasing attacks.<n>Existing detector-evasion methods often lack precise control over semantic preservation.<n>We propose Detector Evasion Policy Optimization (DEPO), a primal-dual reinforcement learning algorithm with a novel GRPO-style group-based policy update.
  arXiv  Detail & Related papers  (2026-05-29T22:19:28Z)
• AIPO: Learning to Reason from Active Interaction [54.10819421625103]
  AIPO enables the policy model to proactively consult three functional collaborative agents when encountering reasoning bottlenecks.<n>AIPO consistently improves reasoning performance, generalizes robustly across different policy models and RLVR algorithms, and effectively expands the reasoning capability boundary of the policy model.
  arXiv  Detail & Related papers  (2026-05-08T19:06:55Z)
• TAG: Target-Agnostic Guidance for Stable Object-Centric Inference in Vision-Language-Action Models [59.13964209628383]
  Vision-Language--Action (VLA) policies have shown strong progress in mapping language instructions and visual observations to robotic actions, yet their reliability degrades in cluttered scenes with distractors.<n>We propose TAG (Target-Agnostic Guidance), a simple inference-time guidance mechanism that explicitly reduces distractor- and appearance-induced bias in VLA policies.<n>We evaluate TAG on standard manipulation benchmarks, including LIBERO, LIBERO-Plus, and VLABench, where it consistently improves robustness under clutter and reduces near-miss and wrong-object executions.
  arXiv  Detail & Related papers  (2026-03-25T17:56:32Z)
• The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis [24.51410516475904]
  This SoK presents a comprehensive overview of the Prompt Injection (PI) landscape, covering attacks, defenses, and their evaluation practices.<n>We introduce AgentPI, a new benchmark designed to systematically evaluate agent behavior under context-dependent interaction settings.<n>We show that many defenses appear effective under existing benchmarks by suppressing contextual inputs, yet fail to generalize to realistic agent settings where context-dependent reasoning is essential.
  arXiv  Detail & Related papers  (2026-02-11T02:47:10Z)
• NeuroFilter: Privacy Guardrails for Conversational LLM Agents [50.75206727081996]
  This work addresses the computational challenge of enforcing privacy for agentic Large Language Models (LLMs)<n>NeuroFilter is a guardrail framework that operationalizes contextual integrity by mapping norm violations to simple directions in the model's activation space.<n>A comprehensive evaluation across over 150,000 interactions, covering models from 7B to 70B parameters, illustrates the strong performance of NeuroFilter.
  arXiv  Detail & Related papers  (2026-01-21T05:16:50Z)
• Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs [21.5603664964501]
  We propose a training-free and efficient method that treats policy violation detection as an out-of-distribution detection problem.<n>Inspired by whitening techniques, we apply a linear transformation to decorrelate the model's hidden activations and standardize them to zero mean and unit variance.<n>On a challenging policy benchmark, our approach achieves state-of-the-art results, surpassing both existing guardrails and fine-tuned reasoning models.
  arXiv  Detail & Related papers  (2025-12-03T17:23:39Z)
• eBPF-PATROL: Protective Agent for Threat Recognition and Overreach Limitation using eBPF in Containerized and Virtualized Environments [0.23332469289621782]
  eBPF-PATROL is a lightweight runtime security agent that monitors and enforces policies in containerized environments.<n>eBPF-PATROL detects and prevents real-time boundary violations, such as reverse shells, privilege escalation, and container escape attempts.
  arXiv  Detail & Related papers  (2025-11-22T18:51:36Z)
• Contamination Detection for VLMs using Multi-Modal Semantic Perturbation [73.76465227729818]
  Open-source Vision-Language Models (VLMs) have achieved state-of-the-art performance on benchmark tasks.<n>Pretraining corpora raise a critical concern for both practitioners and users: inflated performance due to test-set leakage.<n>We show that existing detection approaches either fail outright or exhibit inconsistent behavior.<n>We propose a novel simple yet effective detection method based on multi-modal semantic perturbation.
  arXiv  Detail & Related papers  (2025-11-05T18:59:52Z)
• Defending against Indirect Prompt Injection by Instruction Detection [109.30156975159561]
  InstructDetector is a novel detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks.<n>InstructDetector achieves a detection accuracy of 99.60% in the in-domain setting and 96.90% in the out-of-domain setting, and reduces the attack success rate to just 0.03% on the BIPIA benchmark.
  arXiv  Detail & Related papers  (2025-05-08T13:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.