PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection

• URL: http://arxiv.org/abs/2605.24171v1
• Date: Fri, 22 May 2026 19:44:51 GMT
• Title: PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection
• Abstract summary: We evaluate accuracy, recall, abstention, coverage, and effective F1 using five prompting strategies across five open-weight models on 1,000 CVEs.<n>We find that standard chain-of-thought prompting achieves the strongest overall operational performance.<n> adaptive chain-of-thought frequently suppresses recall and self-consistency induces excessive abstention.
• Score: 11.513151833477837
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models are increasingly used for vulnerability detection, yet their reliability under different prompt formulations remains uncharacterized. We present PromptAudit, a controlled evaluation framework that isolates prompt effects by fixing the dataset, decoding, and parsing while varying only the prompting strategy. Using five prompting strategies across five open-weight models on 1,000 CVEs (6,074 code samples spanning 16 programming languages), we evaluate accuracy, recall, abstention, coverage, and effective F1. We find that standard chain-of-thought prompting achieves the strongest overall operational performance, while few-shot prompting provides model-dependent benefits that are most pronounced for prompt-sensitive models. In contrast, adaptive chain-of-thought frequently suppresses recall and self-consistency induces excessive abstention, sharply reducing effective performance. These results show that vulnerability detection behavior is jointly determined by the model and the prompt, and that prompt sensitivity is a first-class system property that must be explicitly characterized in evaluation and deployment.

Related papers

• Detecting Contaminated Code-Generation Prompt Batches via Influence Functions [4.185678330063222]
  Large language models (LLMs) are increasingly used for code generation, yet they remain vulnerable to prompts that elicit insecure implementations.<n>We propose CodeSIFT, a threat-model-agnostic detection method that leverages influence functions to identify batches of prompts that induce anomalous model behavior.
  arXiv  Detail & Related papers  (2026-08-14T13:37:53Z)
• Prompt Injection Detection is Regime-Dependent: A Deployment-Aware Evaluation with Interpretable Structural Signals [11.542407696902115]
  We present a deployment-aware evaluation of prompt injection detection using a multi-model and multi-regime experimental framework.<n>We compare lexical, semantic, structural, and transformer-based detectors across multiple out-of-distribution settings.<n>Results show that detection performance is highly regime-dependent and sensitive to threshold selection.
  arXiv  Detail & Related papers  (2026-05-26T13:19:25Z)
• TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models [76.92890872338235]
  Large-scale pre-trained Vision-Language models (VLMs) exhibit strong zero-shot generalization, yet are highly vulnerable to imperceptible adversarial perturbations.<n>To enhance robustness without requiring downstream task-specific retraining, we propose TAME, a novel test-time defense.
  arXiv  Detail & Related papers  (2026-05-17T18:07:08Z)
• PromptCD: Test-Time Behavior Enhancement via Polarity-Prompt Contrastive Decoding [85.22047087898311]
  We introduce Polarity-Prompt Contrastive Decoding (PromptCD), a test-time behavior control method that generalizes contrastive decoding to broader enhancement settings.<n>PromptCD constructs paired positive and negative guiding prompts for a target behavior and contrasts model responses to reinforce desirable outcomes.<n>Experiments on the "3H" alignment objectives demonstrate consistent and substantial improvements, indicating that post-trained models can achieve meaningful self-enhancement purely at test time.
  arXiv  Detail & Related papers  (2026-02-24T08:56:52Z)
• MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution [28.062506040151153]
  Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to two key limitations.<n>The heterogeneity of vulnerability patterns undermines the effectiveness of a single unified model, and manual prompt engineering for massive weakness categories is unscalable.<n>We propose textbfMulVul, a retrieval-augmented multi-agent framework for precise and broad-coverage vulnerability detection.
  arXiv  Detail & Related papers  (2026-01-26T12:43:10Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awareness [6.071703608560761]
  Large Language Models (LLMs) often exhibit significant behavioral shifts when they perceive a change from a real-world deployment context to a controlled evaluation setting.<n>This discrepancy poses a critical challenge for AI alignment, as benchmark performance may not accurately reflect a model's true safety and honesty.<n>We introduce a methodology that uses a linear probe to score prompts on a continuous scale from "test-like" to "deploy-like"
  arXiv  Detail & Related papers  (2025-08-30T19:03:14Z)
• Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks [0.5218155982819203]
  Large Language Models (LLMs) are increasingly used as code assistants.<n>This study examines a more direct threat: open-source LLMs generating vulnerable code when prompted.
  arXiv  Detail & Related papers  (2025-07-14T08:36:26Z)
• It Only Gets Worse: Revisiting DL-Based Vulnerability Detectors from a Practical Perspective [14.271145160443462]
  VulTegra compares scratch-trained and pre-trained DL models for vulnerability detection.<n>State-of-the-art (SOTA) detectors still suffer from low consistency, limited real-world capabilities, and scalability challenges.
  arXiv  Detail & Related papers  (2025-07-13T08:02:56Z)
• Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient [50.15090865963094]
  We introduce PertBench, a comprehensive benchmark dataset that includes a wide range of input perturbations.<n>Our analysis reveals substantial vulnerabilities in existing prompt generation strategies.<n>We propose PGO, a gradient-free prompt generation framework that leverages perturbation types as pseudo-gradient signals.
  arXiv  Detail & Related papers  (2024-12-24T06:05:08Z)
• TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models [53.91006249339802]
  We propose a novel defense method called Test-Time Adversarial Prompt Tuning (TAPT) to enhance the inference robustness of CLIP against visual adversarial attacks. TAPT is a test-time defense method that learns defensive bimodal (textual and visual) prompts to robustify the inference process of CLIP. We evaluate the effectiveness of TAPT on 11 benchmark datasets, including ImageNet and 10 other zero-shot datasets.
  arXiv  Detail & Related papers  (2024-11-20T08:58:59Z)
• On the Worst Prompt Performance of Large Language Models [93.13542053835542]
  Performance of large language models (LLMs) is acutely sensitive to the phrasing of prompts. We introduce RobustAlpacaEval, a new benchmark that consists of semantically equivalent case-level queries. Experiments on RobustAlpacaEval with ChatGPT and six open-source LLMs from the Llama, Mistral, and Gemma families uncover substantial variability in model performance.
  arXiv  Detail & Related papers  (2024-06-08T13:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.