When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications

• URL: http://arxiv.org/abs/2605.10176v1
• Date: Mon, 11 May 2026 08:28:45 GMT
• Title: When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications
• Abstract summary: Large language models (LLMs) enable users to query data using conversational input rather than formal query languages such assql.<n>Malicious users can exploit these mechanisms by crafting adversarial prompts that manipulate model behavior and generate unsafe queries.<n>We propose a multi-layered security framework designed to detect and mitigate LLM-mediatedsql injection attacks.
• Score: 4.213098466542674
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Natural language interfaces to structured databases are becoming increasingly common, largely due to advances in large language models (LLMs) that enable users to query data using conversational input rather than formal query languages such as SQL. While this paradigm significantly improves usability and accessibility, it introduces new security risks, particularly the amplification of SQL injection vulnerabilities through the prompt-to-SQL translation process. Malicious users can exploit these mechanisms by crafting adversarial prompts that manipulate model behavior and generate unsafe queries. In this work, we propose a multi-layered security framework designed to detect and mitigate LLM-mediated SQL injection attacks. The framework integrates a front-end security shield for prompt sanitization, an advanced threat detection model for behavioral and semantic anomaly identification, and a signature-based control layer for known attack patterns. We evaluate the proposed framework under diverse and realistic attack scenarios, including prompt injection, obfuscated SQL payloads, and context-manipulation attacks. To ensure robustness, we generate and curate a comprehensive benchmark dataset of adversarial prompts and assess performance across a fine-tuned LLM configuration. Experimental results demonstrate that the proposed approach achieves high detection accuracy while maintaining low false-positive rates, significantly improving the secure deployment of LLM-powered database applications.

Related papers

• An Empirical Evaluation of Prompt Injection Vulnerabilities in Large Language Models Across Multilingual and Obfuscated Attack Scenarios [0.3499870393443268]
  Large Language Models (LLMs) have rapidly evolved, transforming industries by automating complex tasks and generating human-like content.<n>As their adoption accelerates, prompt injection vulnerabilities have become increasingly apparent.<n>This paper presents an empirical evaluation of six state-of-the-art LLMs under diverse adversarial prompt scenarios.
  arXiv  Detail & Related papers  (2026-06-28T20:56:57Z)
• Robust Multimodal Safety via Conditional Decoding [52.92816441364308]
  Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions.<n>We propose a simple conditional decoding strategy, CASA, that utilizes internal representations of MLLMs to predict a binary safety token before response generation.
  arXiv  Detail & Related papers  (2026-03-31T23:19:50Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• SAFENLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces [12.135290721799421]
  We propose textscSafeNlidb, a novel privacy-security alignment framework for NLIDB.<n>The framework features an automated pipeline that generates hybrid chain-of-thought interaction data from scratch.<n>Our method outperforms larger-scale LLMs and ideal-setting baselines.
  arXiv  Detail & Related papers  (2025-11-10T07:05:59Z)
• Are Your LLM-based Text-to-SQL Models Secure? Exploring SQL Injection via Backdoor Attacks [23.036091420064235]
  Security concerns remain largely unexplored, particularly the threat of backdoor attacks.<n>We present Toxic, a novel backdoor attack framework.<n>We demonstrate that injecting only 0.44% of poisoned data can result in an attack success rate of 79.41%, posing a significant risk to database security.
  arXiv  Detail & Related papers  (2025-03-07T14:16:48Z)
• SecAlign: Defending Against Prompt Injection with Preference Optimization [52.48001255555192]
  Adversarial prompts can be injected into external data sources to override the system's intended instruction and execute a malicious instruction.<n>We propose a new defense called SecAlign based on the technique of preference optimization.<n>Our method reduces the success rates of various prompt injections to 10%, even against attacks much more sophisticated than ones seen during training.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Context [49.13497493053742]
  This research explores converting a nonsensical suffix attack into a sensible prompt via a situation-driven contextual re-writing. We combine an independent, meaningful adversarial insertion and situations derived from movies to check if this can trick an LLM. Our approach demonstrates that a successful situation-driven attack can be executed on both open-source and proprietary LLMs.
  arXiv  Detail & Related papers  (2024-07-19T19:47:26Z)
• AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting [54.931241667414184]
  We propose textbfAdaptive textbfShield Prompting, which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks. Our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks.
  arXiv  Detail & Related papers  (2024-03-14T15:57:13Z)
• Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models [79.0183835295533]
  We introduce the first benchmark for indirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities.<n>Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content.<n>We propose two novel defense mechanisms-boundary awareness and explicit reminder-to address these vulnerabilities in both black-box and white-box settings.
  arXiv  Detail & Related papers  (2023-12-21T01:08:39Z)
• From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application? [4.361862281841999]
  We present a comprehensive examination of P$$ injections targeting web applications based on the Langchain framework.<n>Our findings indicate that LLM-integrated applications based on Langchain are highly susceptible to P$$ injection attacks, warranting the adoption of robust defenses.<n>We propose four effective defense techniques that can be integrated as extensions to the Langchain framework.
  arXiv  Detail & Related papers  (2023-08-03T19:03:18Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)
• On the Security Vulnerabilities of Text-to-SQL Models [34.749129843281196]
  We show that modules within six commercial applications can be manipulated to produce malicious code. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms.
  arXiv  Detail & Related papers  (2022-11-28T14:38:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.