SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models

• URL: http://arxiv.org/abs/2502.00847v1
• Date: Sun, 02 Feb 2025 16:40:21 GMT
• Title: SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models
• Authors: Jiawen Zhang, Kejia Chen, Zunlei Feng, Jian Lou, Mingli Song, Jian Liu, Xiaohu Yang,
• Abstract summary: We are among the first attempts towards robust and private LLM inference by tightly integrating two disconnected fields: private inference and prompt ensembling.<n>We propose SecPE, which designs efficient fully homomorphic encryption (FHE) counterparts for the core algorithmic building blocks of prompt ensembling.<n>Results show that SecPE maintains high clean accuracy and offers better robustness at the expense of merely $2.5%$ efficiency overhead.
• Score: 40.1319651615647
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the growing popularity of LLMs among the general public users, privacy-preserving and adversarial robustness have become two pressing demands for LLM-based services, which have largely been pursued separately but rarely jointly. In this paper, to the best of our knowledge, we are among the first attempts towards robust and private LLM inference by tightly integrating two disconnected fields: private inference and prompt ensembling. The former protects users' privacy by encrypting inference data transmitted and processed by LLMs, while the latter enhances adversarial robustness by yielding an aggregated output from multiple prompted LLM responses. Although widely recognized as effective individually, private inference for prompt ensembling together entails new challenges that render the naive combination of existing techniques inefficient. To overcome the hurdles, we propose SecPE, which designs efficient fully homomorphic encryption (FHE) counterparts for the core algorithmic building blocks of prompt ensembling. We conduct extensive experiments on 8 tasks to evaluate the accuracy, robustness, and efficiency of SecPE. The results show that SecPE maintains high clean accuracy and offers better robustness at the expense of merely $2.5\%$ efficiency overhead compared to baseline private inference methods, indicating a satisfactory ``accuracy-robustness-efficiency'' tradeoff. For the efficiency of the encrypted Argmax operation that incurs major slowdown for prompt ensembling, SecPE is 35.4x faster than the state-of-the-art peers, which can be of independent interest beyond this work.

Related papers

• Towards Privacy-Preserving LLM Inference via Collaborative Obfuscation (Technical Report) [11.45459291503604]
  AloePri is the first privacy-preserving LLM inference method for industrial applications.<n>We show that AloePri causes accuracy loss of 0.0%3.5% and exhibits efficiency equivalent to that of plaintext inference.
  arXiv  Detail & Related papers  (2026-03-02T06:16:36Z)
• Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencoder [59.89996751196727]
  Sparse autoencoders (SAEs) have emerged as a powerful tool for interpreting large language models.<n>SAEs' hidden layers have high dimensionality to satisfy sparsity constraints, resulting in prohibitive training and inference costs.<n>Recent Mixture of Experts (MoE) approaches attempt to address this by SAEs into narrower expert networks with gated activation.<n>We propose two key innovations: (1) Multiple Expert Activation that simultaneously engages semantically weighted expert subsets to encourage specialization, and (2) Feature Scaling that enhances diversity through adaptive high-frequency scaling.
  arXiv  Detail & Related papers  (2025-11-07T22:19:34Z)
• One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning [52.966712416640085]
  We propose SMoPE, a novel framework that integrates the benefits of both task-specific and shared prompt strategies.<n>SMoPE consistently outperforms task-specific prompt methods and achieves performance competitive with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-09-29T08:54:58Z)
• SecFwT: Efficient Privacy-Preserving Fine-Tuning of Large Language Models Using Forward-Only Passes [37.63828228378461]
  Large language models (LLMs) have transformed numerous fields, yet their adaptation to specialized tasks in privacy-sensitive domains, such as healthcare and finance, is constrained by the scarcity of accessible training data due to stringent privacy requirements.<n>Secure multi-party computation (MPC)-based privacy-preserving machine learning offers a powerful approach to protect both model parameters and user data.<n>We propose SecFwT, the first MPC-based framework designed for efficient, privacy-preserving LLM fine-tuning.
  arXiv  Detail & Related papers  (2025-06-18T09:36:57Z)
• SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks [17.77094760401298]
  We study the vulnerability of fine-tuned large language models to membership inference attacks (MIAs)<n>We propose SOFT, a novel defense technique that mitigates privacy leakage by leveraging influential data selection with an adjustable parameter to balance utility preservation and privacy protection.
  arXiv  Detail & Related papers  (2025-06-12T07:23:56Z)
• Urania: Differentially Private Insights into AI Use [104.7449031243196]
  $Urania$ provides end-to-end privacy protection by leveraging DP tools such as clustering, partition selection, and histogram-based summarization.<n>Results show the framework's ability to extract meaningful conversational insights while maintaining stringent user privacy.
  arXiv  Detail & Related papers  (2025-06-05T07:00:31Z)
• Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity [21.74620410396962]
  Secure multi-party computation (MPC) is a promising solution to protect the privacy in LLM inference.<n>MPC requires frequent inter-server communication, causing high performance overhead.<n>We propose an efficient private inference system, Comet, which employs an accurate and fast predictor to predict the sparsity distribution of activation output.<n>Comet achieves a 1.87x-2.63x speedup and a 1.94x-2.64x communication reduction.
  arXiv  Detail & Related papers  (2025-05-12T05:29:30Z)
• Prompt Inversion Attack against Collaborative Inference of Large Language Models [14.786666134508645]
  We introduce the concept of prompt inversion attack (PIA), where a malicious participant intends to recover the input prompt through the activation transmitted by its previous participant. Our method achieves an 88.4% token accuracy on the Skytrax dataset with the Llama-65B model when inverting the maximum number of transformer layers.
  arXiv  Detail & Related papers  (2025-03-12T03:20:03Z)
• Federated Fine-Tuning of LLMs: Framework Comparison and Research Directions [59.5243730853157]
  Federated learning (FL) provides a privacy-preserving solution for fine-tuning pre-trained large language models (LLMs) using distributed private datasets. This article conducts a comparative analysis of three advanced federated LLM (FedLLM) frameworks that integrate knowledge distillation (KD) and split learning (SL) to mitigate these issues.
  arXiv  Detail & Related papers  (2025-01-08T11:37:06Z)
• Privacy-preserved LLM Cascade via CoT-enhanced Policy Learning [14.51198171282123]
  Large Language Models (LLMs) have gained significant attention in on-device applications due to their remarkable performance across real-world tasks. We propose a novel Chain-of-Thought (CoT)-enhanced textbfpolicy learning framework for textbfpreserved textbfdeferral decision-making.
  arXiv  Detail & Related papers  (2024-10-10T15:09:52Z)
• The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems [26.528288876732617]
  A set of new timing side channels can be exploited to infer confidential system prompts and those issued by other users. These vulnerabilities echo security challenges observed in traditional computing systems. We propose a token-by-token search algorithm to efficiently recover shared prompt prefixes in the caches.
  arXiv  Detail & Related papers  (2024-09-30T06:55:00Z)
• A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems [67.52782366565658]
  State-of-the-art recommender systems (RSs) depend on categorical features, which ecoded by embedding vectors, resulting in excessively large embedding tables. Despite the prosperity of lightweight embedding-based RSs, a wide diversity is seen in evaluation protocols. This study investigates various LERS' performance, efficiency, and cross-task transferability via a thorough benchmarking process.
  arXiv  Detail & Related papers  (2024-06-25T07:45:00Z)
• FedCoT: Federated Chain-of-Thought Distillation for Large Language Models [24.624093188197126]
  Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks.<n>Small Language Models (SLMs) offer computational efficiency but often lag in performance.<n>We propose FedCoT, a framework designed for the Chain-of-Thought (CoT) distillation of knowledge from LLMs to SLMs.
  arXiv  Detail & Related papers  (2024-06-18T08:48:14Z)
• Efficient Adversarial Training in LLMs with Continuous Attacks [99.5882845458567]
  Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses. C-AdvIPO is an adversarial variant of IPO that does not require utility data for adversarially robust alignment.
  arXiv  Detail & Related papers  (2024-05-24T14:20:09Z)
• TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data [50.797729676285876]
  We propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm.
  arXiv  Detail & Related papers  (2024-02-16T16:41:14Z)
• Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus [99.33091772494751]
  Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields. LLMs are prone to hallucinate untruthful or nonsensical outputs that fail to meet user expectations. We propose a novel reference-free, uncertainty-based method for detecting hallucinations in LLMs.
  arXiv  Detail & Related papers  (2023-11-22T08:39:17Z)
• Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications [63.29358103217275]
  Large Language Models (LLMs) often leads to reduced performance, especially for knowledge-intensive tasks. We propose two conjectures on the nature of the damage: one is certain knowledge being forgotten (or erased) after compression. We introduce a variant called Inference-time Dynamic Prompting (IDP) that can effectively increase prompt diversity without incurring any inference overhead.
  arXiv  Detail & Related papers  (2023-10-02T03:12:06Z)
• An Efficient and Multi-private Key Secure Aggregation for Federated Learning [41.29971745967693]
  We propose an efficient and multi-private key secure aggregation scheme for federated learning. Specifically, we skillfully modify the variant ElGamal encryption technique to achieve homomorphic addition operation. For the high dimensional deep model parameter, we introduce a super-increasing sequence to compress multi-dimensional data into 1-D.
  arXiv  Detail & Related papers  (2023-06-15T09:05:36Z)
• When approximate design for fast homomorphic computation provides differential privacy guarantees [0.08399688944263842]
  Differential privacy (DP) and cryptographic primitives are popular countermeasures against privacy attacks. In this paper, we design SHIELD, a probabilistic approximation algorithm for the argmax operator. Even if SHIELD could have other applications, we here focus on one setting and seamlessly integrate it in the SPEED collaborative training framework.
  arXiv  Detail & Related papers  (2023-04-06T09:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.