SPML: A DSL for Defending Language Models Against Prompt Attacks

• URL: http://arxiv.org/abs/2402.11755v1
• Date: Mon, 19 Feb 2024 00:53:48 GMT
• Title: SPML: A DSL for Defending Language Models Against Prompt Attacks
• Abstract summary: System Prompt Meta Language (SPML) is a domain-specific language for refining prompts and monitoring inputs to large language models (LLMs) SPML actively checks attack prompts, ensuring user inputs align with definitions to prevent malicious execution on the LLM backbone, optimizing costs. We introduce a groundbreaking benchmark with 1.8k system prompts and 20k user inputs, offering the inaugural language and benchmark for chatbots definition evaluation.
• Score: 4.511923587827302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have profoundly transformed natural language applications, with a growing reliance on instruction-based definitions for designing chatbots. However, post-deployment the chatbot definitions are fixed and are vulnerable to attacks by malicious users, emphasizing the need to prevent unethical applications and financial losses. Existing studies explore user prompts' impact on LLM-based chatbots, yet practical methods to contain attacks on application-specific chatbots remain unexplored. This paper presents System Prompt Meta Language (SPML), a domain-specific language for refining prompts and monitoring the inputs to the LLM-based chatbots. SPML actively checks attack prompts, ensuring user inputs align with chatbot definitions to prevent malicious execution on the LLM backbone, optimizing costs. It also streamlines chatbot definition crafting with programming language capabilities, overcoming natural language design challenges. Additionally, we introduce a groundbreaking benchmark with 1.8k system prompts and 20k user inputs, offering the inaugural language and benchmark for chatbot definition evaluation. Experiments across datasets demonstrate SPML's proficiency in understanding attacker prompts, surpassing models like GPT-4, GPT-3.5, and LLAMA. Our data and codes are publicly available at: https://prompt-compiler.github.io/SPML/.

Related papers

• When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Plugins [17.279238456399693]
  We present the first large-scale study of 17 third-party plugins used by over 10,000 public websites.<n>8 of these plugins fail to enforce the integrity of the conversation history transmitted in network requests.<n>We found that 13% of e-commerce websites have already exposed their chatbots to third-party content.
  arXiv  Detail & Related papers  (2025-11-08T02:02:24Z)
• A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms [0.6144680854063939]
  We present a framework for auditing large language models (LLM)-based chatbots for dialect bias.<n>We measure the extent to which they produce quality-of-service harms, which occur when systems do not work equally well for different people.<n>Our framework has three key characteristics that make it useful in practice.
  arXiv  Detail & Related papers  (2025-06-04T19:58:51Z)
• OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Modalities [54.152681077418805]
  Current detection approaches are fallible, and are particularly susceptible to attacks that exploit mismatched generalizations of model capabilities.<n>We propose OMNIGUARD, an approach for detecting harmful prompts across languages and modalities.<n>Our approach improves harmful prompt classification accuracy by 11.57% over the strongest baseline in a multilingual setting.
  arXiv  Detail & Related papers  (2025-05-29T05:25:27Z)
• Type-Constrained Code Generation with Language Models [51.03439021895432]
  We introduce a type-constrained decoding approach that leverages type systems to guide code generation.<n>For this purpose, we develop novel prefix automata and a search over inhabitable types, forming a sound approach to enforce well-typedness on LLM-generated code.<n>Our approach reduces compilation errors by more than half and significantly increases functional correctness in code synthesis, translation, and repair tasks.
  arXiv  Detail & Related papers  (2025-04-12T15:03:00Z)
• Prompt Engineering a Schizophrenia Chatbot: Utilizing a Multi-Agent Approach for Enhanced Compliance with Prompt Instructions [0.0699049312989311]
  Patients with schizophrenia often present with cognitive impairments that may hinder their ability to learn about their condition. While Large Language Models (LLMs) have the potential to make topical mental health information more accessible and engaging, their black-box nature raises concerns about ethics and safety.
  arXiv  Detail & Related papers  (2024-10-10T09:49:24Z)
• Aligning LLMs to Be Robust Against Prompt Injection [55.07562650579068]
  We show that alignment can be a powerful tool to make LLMs more robust against prompt injection attacks. Our method -- SecAlign -- first builds an alignment dataset by simulating prompt injection attacks. Our experiments show that SecAlign robustifies the LLM substantially with a negligible hurt on model utility.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• A test suite of prompt injection attacks for LLM-based machine translation [4.459306403129608]
  LLM-based NLP systems typically work by embedding their input data into prompt templates which contain instructions and/or in-context examples. Recently, Sun and Miceli-Barone proposed a class of PIAs against LLM-based machine translation. We extend this approach to all the language pairs of the WMT 2024 General Machine Translation task.
  arXiv  Detail & Related papers  (2024-10-07T14:01:20Z)
• A Reliable Common-Sense Reasoning Socialbot Built Using LLMs and Goal-Directed ASP [3.17686396799427]
  We propose AutoCompanion, a socialbot that uses an LLM model to translate natural language into predicates. This paper presents the framework design and how an LLM is used to parse user messages and generate a response from the s(CASP) engine output.
  arXiv  Detail & Related papers  (2024-07-26T04:13:43Z)
• Understanding and Mitigating Language Confusion in LLMs [76.96033035093204]
  We evaluate 15 typologically diverse languages with existing and newly-created English and multilingual prompts. We find that Llama Instruct and Mistral models exhibit high degrees of language confusion. We find that language confusion can be partially mitigated via few-shot prompting, multilingual SFT and preference tuning.
  arXiv  Detail & Related papers  (2024-06-28T17:03:51Z)
• On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots [19.423566424346166]
  We study the use of Large Language Model (LLM)-based chatbots to power recommender systems. We observe that the chatbots respond poorly when they encounter under-specified requests. We conjecture that such miscalibrated response tendencies can be attributed to LLM fine-tuning using annotators.
  arXiv  Detail & Related papers  (2024-06-01T15:54:45Z)
• Hidden in Plain Sight: Exploring Chat History Tampering in Interactive Language Models [12.920884182101142]
  Large Language Models (LLMs) have become prevalent in real-world applications, exhibiting impressive text generation performance. To behave interactively, LLM-based chat systems must integrate prior chat history as context into their inputs, following a pre-defined structure. This paper introduces a systematic methodology to inject user-supplied history into LLM conversations without any prior knowledge of the target model.
  arXiv  Detail & Related papers  (2024-05-30T16:36:47Z)
• AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting [54.931241667414184]
  We propose textbfAdaptive textbfShield Prompting, which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks. Our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks.
  arXiv  Detail & Related papers  (2024-03-14T15:57:13Z)
• Measuring and Controlling Instruction (In)Stability in Language Model Dialogs [72.38330196290119]
  System-prompting is a tool for customizing language-model chatbots, enabling them to follow a specific instruction. We propose a benchmark to test the assumption, evaluating instruction stability via self-chats. We reveal a significant instruction drift within eight rounds of conversations. We propose a lightweight method called split-softmax, which compares favorably against two strong baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:10:29Z)
• Which Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code? [51.29970742152668]
  We highlight relying on accuracy-based measurements may lead to an overestimation of models' capabilities. To address these issues, we introduce a technique called SyntaxEval in Syntactic Capabilities.
  arXiv  Detail & Related papers  (2024-01-03T02:44:02Z)
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
  We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs) Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
  arXiv  Detail & Related papers  (2023-10-05T17:01:53Z)
• Neuro-Symbolic Causal Language Planning with Commonsense Prompting [67.06667162430118]
  Language planning aims to implement complex high-level goals by decomposition into simpler low-level steps. Previous methods require either manual exemplars or annotated programs to acquire such ability from large language models. This paper proposes Neuro-Symbolic Causal Language Planner (CLAP) that elicits procedural knowledge from the LLMs with commonsense-infused prompting.
  arXiv  Detail & Related papers  (2022-06-06T22:09:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.