On the Security Vulnerabilities of Text-to-SQL Models

• URL: http://arxiv.org/abs/2211.15363v4
• Date: Sat, 11 May 2024 15:08:58 GMT
• Title: On the Security Vulnerabilities of Text-to-SQL Models
• Authors: Xutan Peng, Yipeng Zhang, Jingfeng Yang, Mark Stevenson,
• Abstract summary: We show that modules within six commercial applications can be manipulated to produce malicious code. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms.
• Score: 34.749129843281196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although it has been demonstrated that Natural Language Processing (NLP) algorithms are vulnerable to deliberate attacks, the question of whether such weaknesses can lead to software security threats is under-explored. To bridge this gap, we conducted vulnerability tests on Text-to-SQL systems that are commonly used to create natural language interfaces to databases. We showed that the Text-to-SQL modules within six commercial applications can be manipulated to produce malicious code, potentially leading to data breaches and Denial of Service attacks. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. In addition, experiments using four open-source language models verified that straightforward backdoor attacks on Text-to-SQL systems achieve a 100% success rate without affecting their performance. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms and encourage exploration of methods to mitigate against them.

Related papers

• MVD: A Multi-Lingual Software Vulnerability Detection Framework [1.0771072841012608]
  We introduce MVD - an innovative multi-lingual vulnerability detection framework. This framework acquires the ability to detect vulnerabilities across multiple languages by concurrently learning from vulnerability data of various languages. Our framework significantly surpasses state-of-the-art methods in multi-lingual vulnerability detection by 83.7% to 193.6% in PR-AUC.
  arXiv  Detail & Related papers  (2024-12-09T02:58:10Z)
• Unmasking Database Vulnerabilities: Zero-Knowledge Schema Inference Attacks in Text-to-SQL Systems [7.613758211231583]
  We introduce a novel zero-knowledge framework for reconstructing the underlying database schema of text-to-generative models without any prior knowledge of the database. We demonstrate that our method achieves high accuracy in reconstructing table names, with F1 scores of up to.99 for generative models and.78 for fine-tuned models. We propose a simple protection mechanism for generative models and empirically show its limitations in mitigating these attacks.
  arXiv  Detail & Related papers  (2024-06-20T17:54:33Z)
• Transforming Computer Security and Public Trust Through the Exploration of Fine-Tuning Large Language Models [0.0]
  "Mallas" are malicious services that exploit large language models (LLMs) for nefarious purposes. This paper delves into the proliferation of Mallas by examining the use of various pre-trained language models and their efficiency and vulnerabilities.
  arXiv  Detail & Related papers  (2024-06-02T06:10:31Z)
• Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models [79.0183835295533]
  We introduce the first benchmark for indirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities. Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content. We propose two novel defense mechanisms-boundary awareness and explicit reminder-to address these vulnerabilities in both black-box and white-box settings.
  arXiv  Detail & Related papers  (2023-12-21T01:08:39Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)
• Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain [17.225973170682604]
  We introduce a generative multi-task model, Unified Text-to-Text Cybersecurity (UTS) UTS is trained on malware reports, phishing site URLs, programming code constructs, social media data, blogs, news articles, and public forum posts. We show UTS improves the performance of some cybersecurity datasets.
  arXiv  Detail & Related papers  (2023-02-20T22:21:26Z)
• CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
  Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
  arXiv  Detail & Related papers  (2023-02-08T11:54:07Z)
• Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution [57.51117978504175]
  Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but produce attacker-specified predictions when the backdoor is activated. We present invisible backdoors that are activated by a learnable combination of word substitution.
  arXiv  Detail & Related papers  (2021-06-11T13:03:17Z)
• Security Vulnerability Detection Using Deep Learning Natural Language Processing [1.4591078795663772]
  We model software vulnerability detection as a natural language processing (NLP) problem with source code treated as texts. For training and testing, we have built a dataset of over 100,000 files in $C$ programming language with 123 types of vulnerabilities. Experiments generate the best performance of over 93% accuracy in detecting security vulnerabilities.
  arXiv  Detail & Related papers  (2021-05-06T01:28:21Z)
• Trojaning Language Models for Fun and Profit [53.45727748224679]
  TROJAN-LM is a new class of trojaning attacks in which maliciously crafted LMs trigger host NLP systems to malfunction. By empirically studying three state-of-the-art LMs in a range of security-critical NLP tasks, we demonstrate that TROJAN-LM possesses the following properties.
  arXiv  Detail & Related papers  (2020-08-01T18:22:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.