An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts

• URL: http://arxiv.org/abs/2501.12521v1
• Date: Tue, 21 Jan 2025 22:24:03 GMT
• Title: An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts
• Abstract summary: We introduce PromptDoctor, a tool specifically designed to detect and correct issues of Dev Prompts.<n>In our analysis of 2,173 Dev Prompts, selected as a representative sample of 40,573 Dev Prompts, we found that 3.46% contained one or more forms of bias.<n> PromptDoctor de-biased 68.29% of the biased Dev Prompts, hardened 41.81% of the vulnerable Dev Prompts, and improved the performance of 37.1% sub-optimal Dev Prompts.
• Score: 2.6254294912127127
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The tidal wave of advancements in Large Language Models (LLMs) has led to their swift integration into application-level logic. Many software systems now use prompts to interact with these black-box models, combining natural language with dynamic values interpolated at runtime, to perform tasks ranging from sentiment analysis to question answering. Due to the programmatic and structured natural language aspects of these prompts, we refer to them as Developer Prompts. Unlike traditional software artifacts, Dev Prompts blend natural language instructions with artificial languages such as programming and markup languages, thus requiring specialized tools for analysis, distinct from classical software evaluation methods. In response to this need, we introduce PromptDoctor, a tool explicitly designed to detect and correct issues of Dev Prompts. PromptDoctor identifies and addresses problems related to bias, vulnerability, and sub-optimal performance in Dev Prompts, helping mitigate their possible harms. In our analysis of 2,173 Dev Prompts, selected as a representative sample of 40,573 Dev Prompts, we found that 3.46% contained one or more forms of bias, 10.75% were vulnerable to prompt injection attacks. Additionally, 3,310 were amenable to automated prompt optimization. To address these issues, we applied PromptDoctor to the flawed Dev Prompts we discovered. PromptDoctor de-biased 68.29% of the biased Dev Prompts, hardened 41.81% of the vulnerable Dev Prompts, and improved the performance of 37.1% sub-optimal Dev Prompts. Finally, we developed a PromptDoctor VSCode extension, enabling developers to easily enhance Dev Prompts in their existing development workflows. The data and source code for this work are available at

Related papers

• REprompt: Prompt Generation for Intelligent Software Development Guided by Requirements Engineering [43.0808976544794]
  Large language models increasingly function as foundation models within coding agents.<n> prompts play a central role in agent-based intelligent software development.<n>We propose REprompt, a multi-agent prompt optimization framework guided by requirements engineering.
  arXiv  Detail & Related papers  (2026-01-23T07:14:34Z)
• Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization [82.29178197694819]
  We derive and evaluate development-specific prompt optimization guidelines.<n>We use an iterative, test-driven approach to automatically refine code generation prompts.<n>We conduct an assessment with 50 practitioners, who report their usage of the elicited prompt improvement patterns.
  arXiv  Detail & Related papers  (2026-01-19T15:01:42Z)
• PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning [55.78158607697319]
  Large language models (LLMs) are evolving from conversational systems into strong reasoners for tasks such as Olympiad mathematics and competitive programming.<n>We present PromptCoT 2.0, a scalable framework that replaces hand-crafted synthesiss with an expectation-maximization loop.<n>This produces problems that are both harder and more diverse than prior corpora.
  arXiv  Detail & Related papers  (2025-09-24T08:46:29Z)
• Understanding Prompt Management in GitHub Repositories: A Call for Best Practices [11.853151610220111]
  The rapid adoption of foundation models has given rise to promptware, i.e., software built using natural language prompts.<n>In this study, we perform an empirical analysis of 24,800 open-source prompts from 92 GitHub repositories to investigate prompt management practices and quality attributes.
  arXiv  Detail & Related papers  (2025-09-15T20:18:22Z)
• Understanding Prompt Programming Tasks and Questions [19.414347044995644]
  We develop a taxonomy of 25 tasks prompt programmers do and 51 questions they ask, measuring the importance of each task and question.<n>We find that prompt programming is not well-supported: all tasks are done manually, and 16 of the 51 questions -- including a majority of the most important ones -- remain unanswered.
  arXiv  Detail & Related papers  (2025-07-23T07:01:44Z)
• Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models [72.4723784999432]
  Large Language Models (LLMs) perform best with well-crafted prompts, yet prompt engineering remains manual, inconsistent, and inaccessible to non-experts.<n>Promptomatix transforms natural language task descriptions into high-quality prompts without requiring manual tuning or domain expertise.<n>System analyzes user intent, generates synthetic training data, selects prompting strategies, and refines prompts using cost-aware objectives.
  arXiv  Detail & Related papers  (2025-07-17T18:18:20Z)
• OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Modalities [54.152681077418805]
  Current detection approaches are fallible, and are particularly susceptible to attacks that exploit mismatched generalizations of model capabilities.<n>We propose OMNIGUARD, an approach for detecting harmful prompts across languages and modalities.<n>Our approach improves harmful prompt classification accuracy by 11.57% over the strongest baseline in a multilingual setting.
  arXiv  Detail & Related papers  (2025-05-29T05:25:27Z)
• Prompting LLMs for Code Editing: Struggles and Remedies [39.02507244469977]
  Large Language Models (LLMs) are rapidly transforming software engineering, with coding assistants embedded in an IDE becoming increasingly prevalent. This paper addresses part of this gap through a multi-phased investigation of developer interactions with an LLM-powered code editing and transformation feature, Transform Code, in an IDE widely used at Google. We analyze telemetry logs of the feature usage, revealing that frequent re-prompting can be an indicator of developer struggles with using Transform Code. We propose and evaluate a tool, AutoPrompter, for automatically improving prompts by inferring missing information from the surrounding code context, leading to a 27% improvement in
  arXiv  Detail & Related papers  (2025-04-28T18:59:28Z)
• Towards Detecting Prompt Knowledge Gaps for Improved LLM-guided Issue Resolution [3.768737590492549]
  We analyze 433 developer-ChatGPT conversations within GitHub issue threads to examine the impact of prompt knowledge gaps and conversation styles on issue resolution.<n>We find that ineffective conversations contain knowledge gaps in 54.7% of prompts, compared to only 13.2% in effective ones.<n>We identify key textual and code relateds-Specificity, Contextual Richness, and Clarity-that are associated with successful issue closure and help assess prompt quality.
  arXiv  Detail & Related papers  (2025-01-20T19:41:42Z)
• Closing the Gap: A User Study on the Real-world Usefulness of AI-powered Vulnerability Detection & Repair in the IDE [5.824774194964031]
  We implement a vulnerability detection and fix tool with professional software developers on real projects that they own.<n>DeepVulGuard scans code for vulnerabilities, suggests fixes, provides natural-language explanations for alerts and fixes, leveraging chat interfaces.<n>We find that although state-of-the-art AI-powered detection and fix tools show promise, they are not yet practical for real-world use due to a high rate of false positives and non-applicable fixes.
  arXiv  Detail & Related papers  (2024-12-18T20:19:56Z)
• Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts [12.979519344826748]
  We argue that some prompts are programs and that the development of prompts is a distinct phenomenon in programming known as "prompt programming" We develop an understanding of prompt programming using Straussian grounded theory through interviews with 20 developers engaged in prompt development.
  arXiv  Detail & Related papers  (2024-09-19T03:55:49Z)
• Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermark [4.071771108540602]
  Visual Prompt Learning (VPL) differs from traditional fine-tuning methods in reducing significant resource consumption by avoiding updating pre-trained model parameters. Developers of Visual Prompts as a Service (V) have emerged. These developers profit by providing well-crafted prompts to authorized customers. However, a significant drawback is that prompts can be easily copied and redistributed, threatening the intellectual property of V developers.
  arXiv  Detail & Related papers  (2024-05-24T02:31:03Z)
• AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs [51.217126257318924]
  We present a novel method that uses another Large Language Models, called the AdvPrompter, to generate human-readable adversarial prompts in seconds. We train the AdvPrompter using a novel algorithm that does not require access to the gradients of the TargetLLM. The trained AdvPrompter generates suffixes that veil the input instruction without changing its meaning, such that the TargetLLM is lured to give a harmful response.
  arXiv  Detail & Related papers  (2024-04-21T22:18:13Z)
• Efficient Prompting Methods for Large Language Models: A Survey [50.82812214830023]
  Efficient Prompting Methods have attracted a wide range of attention.<n>We discuss Automatic Prompt Engineering for different prompt components and Prompt Compression in continuous and discrete spaces.
  arXiv  Detail & Related papers  (2024-04-01T12:19:08Z)
• PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts [76.18347405302728]
  This study uses a plethora of adversarial textual attacks targeting prompts across multiple levels: character, word, sentence, and semantic. The adversarial prompts are then employed in diverse tasks including sentiment analysis, natural language inference, reading comprehension, machine translation, and math problem-solving. Our findings demonstrate that contemporary Large Language Models are not robust to adversarial prompts.
  arXiv  Detail & Related papers  (2023-06-07T15:37:00Z)
• Ignore Previous Prompt: Attack Techniques For Language Models [0.0]
  We propose PromptInject, a framework for mask-based adversarial prompt composition. We show how GPT-3, the most widely deployed language model in production, can be easily misaligned by simple handcrafted inputs.
  arXiv  Detail & Related papers  (2022-11-17T13:43:20Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• Bayesian Prompt Learning for Image-Language Model Generalization [64.50204877434878]
  We use the regularization ability of Bayesian methods to frame prompt learning as a variational inference problem. Our approach regularizes the prompt space, reduces overfitting to the seen prompts and improves the prompt generalization on unseen prompts. We demonstrate empirically on 15 benchmarks that Bayesian prompt learning provides an appropriate coverage of the prompt space.
  arXiv  Detail & Related papers  (2022-10-05T17:05:56Z)
• Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models [116.25562358482962]
  State-of-the-art neural language models can be used to solve ad-hoc language tasks without the need for supervised training. PromptIDE allows users to experiment with prompt variations, visualize prompt performance, and iteratively optimize prompts.
  arXiv  Detail & Related papers  (2022-08-16T17:17:53Z)
• PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts [106.82620362222197]
  PromptSource is a system for creating, sharing, and using natural language prompts. Prompts are functions that map an example from a dataset to a natural language input and target output. Over 2,000 prompts for roughly 170 datasets are already available in PromptSource.
  arXiv  Detail & Related papers  (2022-02-02T20:48:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.