PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents

• URL: http://arxiv.org/abs/2608.02687v3
• Date: Thu, 06 Aug 2026 01:19:44 GMT
• Title: PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents
• Abstract summary: Existing data loss prevention solutions rely on rigid patterns, model fine-tuning, or vendor-managed classifiers with customizability.<n>We present PolicyGuard, a pre-model interception framework that classifies prompts using a natural language policy file.<n>Our key contributions are: (1) the policy-as-prompt paradigm, where classification criteria are defined entirely in a policy document editable by non-engineers without code changes or model retraining; and (2) a sealed evaluation protocol with template-family-level data splits, hidden holdouts, and frozen test sets to rigorously assess generalization.
• Score: 0.8836202788658859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI coding agents accept free-form natural language prompts that may inadvertently contain credentials, personally identifiable information (PII), or proprietary business data. Existing data loss prevention (DLP) solutions rely on rigid regex patterns, model fine-tuning, or vendor-managed classifiers with limited customizability. We present PolicyGuard, a pre-model interception framework that classifies user prompts using an LLM guided by a natural language policy file. Our key contributions are: (1) the policy-as-prompt paradigm, where DLP classification criteria are defined entirely in a plaintext policy document editable by non-engineers without code changes or model retraining; (2) a sealed evaluation protocol with template-family-level data splits, hidden holdouts, and frozen test sets to rigorously assess generalization; and (3) a comprehensive empirical evaluation across 2,000 multilingual prompts demonstrating 96.5% effective block rate (EBR) with only 3.0% false positive rate (FPR) on a frozen test set of 927 prompts, and perfect 100% accuracy on a 217-prompt hidden holdout. Information-matched baseline experiments show that PolicyGuard's natural language format significantly outperforms equivalent content in JSON format (McNemar chi-squared = 31.58, p < 0.001) and dramatically outperforms zero-shot classification (Cohen's h = 0.915). Cross-model portability experiments demonstrate that the same policy achieves 86.4-96.5% EBR across four different LLMs without modification.

Related papers

• Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis [31.25049774595995]
  We study whether refusal behavior remains intact (Aligned) or has been refusalablated (Abliterated) within same-lineage models.<n>We evaluate Aligned and Abliterated states on vulnerability detection, CWE attribution, vulnerableline localization, root-cause localization, and executable patch validation.
  arXiv  Detail & Related papers  (2026-07-07T05:04:48Z)
• Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis [0.0]
  Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation.<n>This paper provides the first systematic investigation of whether LLMs can identify the model family behind political analysis texts under anonymization conditions.
  arXiv  Detail & Related papers  (2026-05-20T04:56:42Z)
• How Code Representation Shapes False-Positive Dynamics in Cross-Language LLM Vulnerability Detection [8.360057179027322]
  How code representation format shapes false positive behaviour in cross-language vulnerability detection remains poorly understood.<n>We systematically vary training intensity and code representation format, comparing raw source text with pruned Abstract Syntax Trees at both training time and inference time.<n>Cross-language FPR reflects the joint effect of training-time and inference-time representation, not either alone.
  arXiv  Detail & Related papers  (2026-04-30T11:01:03Z)
• LLM-Redactor: An Empirical Evaluation of Eight Techniques for Privacy-Preserving LLM Requests [0.0]
  Coding agents and LLM-powered applications routinely send potentially sensitive content to cloud LLM APIs where it may be logged, retained, used for training, or subpoenaed.<n>We present a systematic empirical evaluation of eight techniques for privacy-preserving LLM requests.<n>We implement all eight (or a tractable research-stage subset where deployment is not yet feasible) in an open-source shim compatible with MCP and any OpenAI-compatible API.
  arXiv  Detail & Related papers  (2026-04-13T21:05:42Z)
• CLASP: Defending Hybrid Large Language Models Against Hidden State Poisoning Attacks [48.54598003197356]
  State space models (SSMs) like Mamba have gained significant traction as efficient alternatives to Transformers.<n>HiSPAs, a recently discovered vulnerability that corrupts SSM memory through adversarial strings, pose a critical threat to these architectures.<n>We introduce the CLASP model to defend against this threat.
  arXiv  Detail & Related papers  (2026-03-12T17:29:55Z)
• Self-reflection in Automated Qualitative Coding: Improving Text Annotation through Secondary LLM Critique [1.5749416770494706]
  Large language models (LLMs) allow for sophisticated qualitative coding of large datasets.<n>We present a simple, generalizable two-stage workflow: an LLM applies a human-designed codebook; a secondary LLM critic performs self-reflection on each positive label.<n>We evaluate this approach on six qualitative codes over 3,000 high-content emails from Apache Software Foundation project evaluation discussions.
  arXiv  Detail & Related papers  (2026-01-14T22:27:13Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning [58.16354555208417]
  PAD and FFD are proposed to protect face data from physical media-based Presentation Attacks and digital editing-based DeepFakes, respectively.<n>The lack of a Unified Face Attack Detection model to simultaneously handle attacks in these two categories is mainly attributed to two factors.<n>We present a novel Visual-Language Model-based Hierarchical Prompt Tuning Framework that adaptively explores multiple classification criteria from different semantic spaces.
  arXiv  Detail & Related papers  (2025-05-19T16:35:45Z)
• Self-Evaluation Improves Selective Generation in Large Language Models [54.003992911447696]
  We reformulate open-ended generation tasks into token-level prediction tasks. We instruct an LLM to self-evaluate its answers. We benchmark a range of scoring methods based on self-evaluation.
  arXiv  Detail & Related papers  (2023-12-14T19:09:22Z)
• Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions [50.92702206798324]
  We propose a generative prompting framework for zero-shot text classification. GEN-Z measures the LM likelihood of input text conditioned on natural language descriptions of labels. We show that zero-shot classification with simple contextualization of the data source consistently outperforms both zero-shot and few-shot baselines.
  arXiv  Detail & Related papers  (2023-11-13T07:12:57Z)
• ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction [51.87391234815163]
  E-commerce platforms require structured product data in the form of attribute-value pairs.<n>BERT-based extraction methods require large amounts of task-specific training data.<n>This paper explores using large language models (LLMs) as a more training-data efficient and robust alternative.
  arXiv  Detail & Related papers  (2023-10-19T07:39:00Z)
• SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models [55.60306377044225]
  "SelfCheckGPT" is a simple sampling-based approach to fact-check the responses of black-box models. We investigate this approach by using GPT-3 to generate passages about individuals from the WikiBio dataset.
  arXiv  Detail & Related papers  (2023-03-15T19:31:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.