Confidential Prompting: Privacy-preserving LLM Inference on Cloud

• URL: http://arxiv.org/abs/2409.19134v4
• Date: Mon, 25 Aug 2025 15:50:28 GMT
• Title: Confidential Prompting: Privacy-preserving LLM Inference on Cloud
• Authors: Caihua Li, In Gim, Lin Zhong,
• Abstract summary: We present Obfuscated Secure Partitioned Decoding (OSPD), a system built on two key innovations.<n>OSPD isolates user prompts within per-user processes residing in a confidential virtual machine on the cloud.<n>PO introduces a novel cryptographic technique that enhances SPD resilience against advanced prompt reconstruction attacks.
• Score: 1.8575142641062914
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces a vision of confidential prompting: securing user prompts from untrusted, cloud-hosted large language model (LLM) provider while preserving model confidentiality, output invariance, and compute efficiency. As a first step toward this vision, we present Obfuscated Secure Partitioned Decoding (OSPD), a system built on two key innovations. First, Secure Partitioned Decoding (SPD) isolates user prompts within per-user processes residing in a confidential virtual machine (CVM) on the cloud, which are inaccessible for the cloud LLM while allowing it to generate tokens efficiently. Second, Prompt Obfuscation (PO) introduces a novel cryptographic technique that enhances SPD resilience against advanced prompt reconstruction attacks. Together, these innovations ensure OSPD protects both prompt and model confidentiality while maintaining service functionality. OSPD enables practical, privacy-preserving cloud-hosted LLM inference for sensitive applications, such as processing personal data, clinical records, and financial documents.

Related papers

• Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models [39.390624817461905]
  Talaria is a confidential inference framework that partitions the Large Language Models pipeline to protect client data.<n>Talaria executes sensitive, weight-independent operations within a client-controlled Confidential Virtual Machine.<n>Talaria can defend against state-of-the-art token inference attacks, reducing token reconstruction accuracy from over 97.5% to an average of 1.34%.
  arXiv  Detail & Related papers  (2026-02-27T06:37:07Z)
• NeuroFilter: Privacy Guardrails for Conversational LLM Agents [50.75206727081996]
  This work addresses the computational challenge of enforcing privacy for agentic Large Language Models (LLMs)<n>NeuroFilter is a guardrail framework that operationalizes contextual integrity by mapping norm violations to simple directions in the model's activation space.<n>A comprehensive evaluation across over 150,000 interactions, covering models from 7B to 70B parameters, illustrates the strong performance of NeuroFilter.
  arXiv  Detail & Related papers  (2026-01-21T05:16:50Z)
• Design and Optimization of Cloud Native Homomorphic Encryption Workflows for Privacy-Preserving ML Inference [0.0]
  Homomorphic Encryption (HE) has emerged as a compelling technique that enables cryptographic computation on encrypted data.<n>The integration of HE within large scale cloud native pipelines remains constrained by high computational overhead, orchestration complexity, and model compatibility issues.<n>This paper presents a systematic framework for the design and optimization of cloud native homomorphic encryption that support privacy ML inference.
  arXiv  Detail & Related papers  (2025-10-28T15:13:32Z)
• VFEFL: Privacy-Preserving Federated Learning against Malicious Clients via Verifiable Functional Encryption [3.329039715890632]
  Federated learning is a promising distributed learning paradigm that enables collaborative model training without exposing local client data.<n>The distributed nature of federated learning makes it particularly vulnerable to attacks raised by malicious clients.<n>This paper proposes a privacy-preserving federated learning framework based on verifiable functional encryption.
  arXiv  Detail & Related papers  (2025-06-15T13:38:40Z)
• Urania: Differentially Private Insights into AI Use [104.7449031243196]
  $Urania$ provides end-to-end privacy protection by leveraging DP tools such as clustering, partition selection, and histogram-based summarization.<n>Results show the framework's ability to extract meaningful conversational insights while maintaining stringent user privacy.
  arXiv  Detail & Related papers  (2025-06-05T07:00:31Z)
• Anti-adversarial Learning: Desensitizing Prompts for Large Language Models [13.674984661911607]
  We propose PromptObfus, a novel method for desensitizing LLM prompts.<n>The core idea of PromptObfus is "anti-adversarial" learning, which perturbs privacy words in the prompt to obscure sensitive information.<n>We show that PromptObfus effectively prevents privacy inference from remote LLMs while preserving task performance.
  arXiv  Detail & Related papers  (2025-04-25T06:19:02Z)
• Pr$εε$mpt: Sanitizing Sensitive Prompts for LLMs [49.84954577111077]
  Pr$epsilonepsilon$mpt is a novel system that implements a prompt sanitizer. We show that Pr$epsilonepsilon$mpt is a practical method to achieve meaningful privacy guarantees.
  arXiv  Detail & Related papers  (2025-04-07T14:52:40Z)
• Defeating Prompt Injections by Design [79.00910871948787]
  CaMeL is a robust defense that creates a protective system layer around the Large Language Models.<n>To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query.<n>To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows.
  arXiv  Detail & Related papers  (2025-03-24T15:54:10Z)
• PersGuard: Preventing Malicious Personalization via Backdoor Attacks on Pre-trained Text-to-Image Diffusion Models [51.458089902581456]
  We introduce PersGuard, a novel backdoor-based approach that prevents malicious personalization of specific images.<n>Our method significantly outperforms existing techniques, offering a more robust solution for privacy and copyright protection.
  arXiv  Detail & Related papers  (2025-02-22T09:47:55Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• PFID: Privacy First Inference Delegation Framework for LLMs [34.59282305562392]
  This paper introduces a novel privacy-preservation framework named PFID for LLMs. It addresses critical privacy concerns by localizing user data through model sharding and singular value decomposition.
  arXiv  Detail & Related papers  (2024-06-18T03:27:09Z)
• Privacy-Preserving Deep Learning Using Deformable Operators for Secure Task Learning [14.187385349716518]
  Existing methods for privacy preservation rely on image encryption or perceptual transformation approaches. We propose a novel Privacy-Preserving framework that uses a set of deformable operators for secure task learning.
  arXiv  Detail & Related papers  (2024-04-08T19:46:20Z)
• EmojiCrypt: Prompt Encryption for Secure Communication with Large Language Models [41.090214475309516]
  Cloud-based large language models (LLMs) pose substantial risks of data breaches and unauthorized access to sensitive information. This paper proposes a simple yet effective mechanism EmojiCrypt to protect user privacy.
  arXiv  Detail & Related papers  (2024-02-08T17:57:11Z)
• ConfusionPrompt: Practical Private Inference for Online Large Language Models [3.8134804426693094]
  State-of-the-art large language models (LLMs) are typically deployed as online services, requiring users to transmit detailed prompts to cloud servers. We introduce ConfusionPrompt, a novel framework for private LLM inference that protects user privacy by decomposing the original prompt into smaller sub-prompts. We show that ConfusionPrompt achieves significantly higher utility than local inference methods using open-source models and perturbation-based techniques.
  arXiv  Detail & Related papers  (2023-12-30T01:26:42Z)
• Silent Guardian: Protecting Text from Malicious Exploitation by Large Language Models [63.91178922306669]
  We introduce Silent Guardian, a text protection mechanism against large language models (LLMs) By carefully modifying the text to be protected, TPE can induce LLMs to first sample the end token, thus directly terminating the interaction. We show that SG can effectively protect the target text under various configurations and achieve almost 100% protection success rate in some cases.
  arXiv  Detail & Related papers  (2023-12-15T10:30:36Z)
• DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer [57.04801796205638]
  Large Language Models (LLMs) have emerged as dominant tools for various tasks. However, concerns surrounding data privacy present obstacles due to the tuned prompts' dependency on sensitive private information. We present Differentially-Private Offsite Prompt Tuning (DP-OPT) to address this challenge.
  arXiv  Detail & Related papers  (2023-11-27T02:01:10Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• InferDPT: Privacy-Preserving Inference for Black-box Large Language Model [66.07752875835506]
  InferDPT is the first practical framework for the privacy-preserving Inference of black-box LLMs.<n>RANTEXT is a novel differential privacy mechanism integrated into the perturbation module of InferDPT.
  arXiv  Detail & Related papers  (2023-10-18T18:00:11Z)
• Hide and Seek (HaS): A Lightweight Framework for Prompt Privacy Protection [6.201275002179716]
  We introduce the HaS framework, where "H(ide)" and "S(eek)" represent its two core processes: hiding private entities for anonymization and seeking private entities for de-anonymization. To quantitatively assess HaS's privacy protection performance, we propose both black-box and white-box adversarial models.
  arXiv  Detail & Related papers  (2023-09-06T14:54:11Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)
• Privacy-Preserving Machine Learning in Untrusted Clouds Made Simple [2.3518279773643287]
  We present a practical framework to deploy privacy-preserving machine learning applications in untrusted clouds. We shield unmodified PyTorch ML applications by running them in Intel SGX enclaves with model parameters and encrypted input data. Our approach is completely transparent to the machine learning application: the developer and the end-user do not need to modify the application in any way.
  arXiv  Detail & Related papers  (2020-09-09T16:16:06Z)
• CryptoSPN: Privacy-preserving Sum-Product Network Inference [84.88362774693914]
  We present a framework for privacy-preserving inference of sum-product networks (SPNs) CryptoSPN achieves highly efficient and accurate inference in the order of seconds for medium-sized SPNs.
  arXiv  Detail & Related papers  (2020-02-03T14:49:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.