SecGPT: An Execution Isolation Architecture for LLM-Based Systems
- URL: http://arxiv.org/abs/2403.04960v1
- Date: Fri, 8 Mar 2024 00:02:30 GMT
- Title: SecGPT: An Execution Isolation Architecture for LLM-Based Systems
- Authors: Yuhao Wu, Franziska Roesner, Tadayoshi Kohno, Ning Zhang, Umar Iqbal
- Abstract summary: SecGPT aims to mitigate the security and privacy issues that arise with the execution of third-party apps.
We evaluate SecGPT against a number of case study attacks and demonstrate that it protects against many security, privacy, and safety issues.
- Score: 37.47068167748932
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models (LLMs) extended as systems, such as ChatGPT, have begun
supporting third-party applications. These LLM apps leverage the de facto
natural language-based automated execution paradigm of LLMs: that is, apps and
their interactions are defined in natural language, provided access to user
data, and allowed to freely interact with each other and the system. These LLM
app ecosystems resemble the settings of earlier computing platforms, where
there was insufficient isolation between apps and the system. Because
third-party apps may not be trustworthy, and exacerbated by the imprecision of
the natural language interfaces, the current designs pose security and privacy
risks for users. In this paper, we propose SecGPT, an architecture for
LLM-based systems that aims to mitigate the security and privacy issues that
arise with the execution of third-party apps. SecGPT's key idea is to isolate
the execution of apps and more precisely mediate their interactions outside of
their isolated environments. We evaluate SecGPT against a number of case study
attacks and demonstrate that it protects against many security, privacy, and
safety issues that exist in non-isolated LLM-based systems. The performance
overhead incurred by SecGPT to improve security is under 0.3x for
three-quarters of the tested queries. To foster follow-up research, we release
SecGPT's source code at https://github.com/llm-platform-security/SecGPT.
Related papers
- System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective [24.583984374370342]
Large Language Model-based systems (LLM systems) are information and query processing systems.
We present a system-level defense based on the principles of information flow control that we call an f-secure LLM system.
arXiv Detail & Related papers (2024-09-27T18:41:58Z) - Attacks on Third-Party APIs of Large Language Models [15.823694509708302]
Large language model (LLM) services have recently begun offering a plugin ecosystem to interact with third-party API services.
This innovation enhances the capabilities of LLMs, but it also introduces risks.
This paper proposes a new attacking framework to examine security and safety vulnerabilities within LLM platforms that incorporate third-party services.
arXiv Detail & Related papers (2024-04-24T19:27:02Z) - AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting [54.931241667414184]
We propose textbfAdaptive textbfShield Prompting, which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks.
Our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks.
arXiv Detail & Related papers (2024-03-14T15:57:13Z) - A New Era in LLM Security: Exploring Security Concerns in Real-World
LLM-based Systems [47.18371401090435]
We analyze the security of Large Language Model (LLM) systems, instead of focusing on the individual LLMs.
We propose a multi-layer and multi-step approach and apply it to the state-of-art OpenAI GPT4.
We found that although the OpenAI GPT4 has designed numerous safety constraints to improve its safety features, these safety constraints are still vulnerable to attackers.
arXiv Detail & Related papers (2024-02-28T19:00:12Z) - LM-Polygraph: Uncertainty Estimation for Language Models [71.21409522341482]
Uncertainty estimation (UE) methods are one path to safer, more responsible, and more effective use of large language models (LLMs)
We introduce LM-Polygraph, a framework with implementations of a battery of state-of-the-art UE methods for LLMs in text generation tasks, with unified program interfaces in Python.
It introduces an extendable benchmark for consistent evaluation of UE techniques by researchers, and a demo web application that enriches the standard chat dialog with confidence scores.
arXiv Detail & Related papers (2023-11-13T15:08:59Z) - Identifying and Mitigating Vulnerabilities in LLM-Integrated
Applications [37.316238236750415]
Large language models (LLMs) are increasingly deployed as the service backend for LLM-integrated applications.
In this work, we consider a setup where the user and LLM interact via an LLM-integrated application in the middle.
We identify potential vulnerabilities that can originate from the malicious application developer or from an outsider threat.
We develop a lightweight, threat-agnostic defense that mitigates both insider and outsider threats.
arXiv Detail & Related papers (2023-11-07T20:13:05Z) - SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs)
Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
arXiv Detail & Related papers (2023-10-05T17:01:53Z) - LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins [31.678328189420483]
Large language model (LLM) platforms have recently begun offering an app ecosystem to interface with third-party services on the internet.
While these apps extend the capabilities of LLM platforms, they are developed by arbitrary third parties and thus cannot be implicitly trusted.
We propose a framework that lays a foundation for LLM platform designers to analyze and improve the security, privacy, and safety of current and future third-party integrated LLM platforms.
arXiv Detail & Related papers (2023-09-19T02:20:10Z) - Not what you've signed up for: Compromising Real-World LLM-Integrated
Applications with Indirect Prompt Injection [64.67495502772866]
Large Language Models (LLMs) are increasingly being integrated into various applications.
We show how attackers can override original instructions and employed controls using Prompt Injection attacks.
We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
arXiv Detail & Related papers (2023-02-23T17:14:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.