AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

• URL: http://arxiv.org/abs/2607.28617v2
• Date: Thu, 06 Aug 2026 17:19:16 GMT
• Title: AISPA: User-Centric System Prompt Auditing for Large Language Model Applications
• Abstract summary: System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications.<n>They are used throughout commercial AI products, but are rarely disclosed to the public or regulators.<n>We introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems.
• Score: 39.37840538615966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.

Related papers

• FLARE-AI: Flaw Reporting for AI [51.64618567247884]
  Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety.<n>Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what to report, and groups who receive reports rarely share them with other relevant stakeholders.<n>We audit 12 reporting systems published by AI developers, cybersecurity groups, and AI flaw aggregators.<n>We introduce FLARE-AI, an open-source AI flaw reporting system designed for interoperability with existing systems.
  arXiv  Detail & Related papers  (2026-06-30T12:27:09Z)
• The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents [29.98056047452529]
  Computer-use agents (CUAs) can now autonomously complete complex tasks in real environments, but when misled, they can also be automated.<n>We introduce OSBLIND, a benchmark that evaluates CUAs under unintended attack conditions.<n>Most CUAs exceed 90% attack success rate (ASR), and even safety-aligned Claude 4.5 Sonnet reaches 72.7% ASR.
  arXiv  Detail & Related papers  (2026-04-12T10:52:42Z)
• Who Controls the Conversation? User Perspectives On Generative AI (LLM) System Prompts [7.340845393655051]
  This paper argues that system prompts warrant explicit, user-centred design attention.<n>It asks: what do system prompts contain, how do end-users perceive them, and what do these perceptions offer for design and governance practice?<n>Our results reveal user perspectives on: the benefits and risks of system prompts; the values they prefer to be associated with prompt-design; their levels of comfort with different types of prompts; and degrees of transparency and user control regarding prompt content.
  arXiv  Detail & Related papers  (2026-02-16T17:59:30Z)
• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• The AI Agent Index [8.48525754659057]
  Agentic AI systems can plan and execute complex tasks with limited human involvement.<n>There is currently no structured framework for documenting the technical components, intended uses, and safety features of agentic systems.<n>The AI Agent Index is the first public database to document information about currently deployed agentic AI systems.
  arXiv  Detail & Related papers  (2025-02-03T18:59:13Z)
• Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy [53.54777131440989]
  Large Language Models (LLMs) are susceptible to security and safety threats.<n>One major cause of these vulnerabilities is the lack of an instruction hierarchy.<n>We introduce the instructional segment Embedding (ISE) technique, inspired by BERT, to modern large language models.
  arXiv  Detail & Related papers  (2024-10-09T12:52:41Z)
• Designing Secure AI-based Systems: a Multi-Vocal Literature Review [5.799668199535053]
  We present 16 architectural security guidelines for the design of AI-based systems. The guidelines could support practitioners with actionable advice on the secure development of AI-based systems.
  arXiv  Detail & Related papers  (2024-07-26T08:04:05Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.