CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference

• URL: http://arxiv.org/abs/2412.10652v2
• Date: Tue, 10 Jun 2025 09:20:09 GMT
• Title: CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference
• Authors: Jinglong Luo, Guanzhong Chen, Yehong Zhang, Shiyu Liu, Hui Wang, Yue Yu, Xun Zhou, Yuan Qi, Zenglin Xu,
• Abstract summary: Existing Privacy-Preserving Transformer Inference frameworks face the "impossible" of balancing privacy, efficiency, and performance.<n>We introduce CENTAUR, a groundbreaking framework that seamlessly integrates SMPC and random permutations.<n>Our experiments demonstrate CENTAUR's ability to resist diverse data reconstruction attacks, achieve plaintext-level inference accuracy, and boost inference speed by 5.0-30.4 times.
• Score: 36.22164026463692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the growing deployment of pre-trained models like Transformers on cloud platforms, privacy concerns about model parameters and inference data are intensifying. Existing Privacy-Preserving Transformer Inference (PPTI) frameworks face the "impossible trinity" of balancing privacy, efficiency, and performance: Secure Multi-Party Computation (SMPC)-based approaches ensure strong privacy but suffer from high computational overhead and performance losses; Conversely, permutation-based methods achieve near-plaintext efficiency and accuracy but compromise privacy by exposing sensitive model parameters and intermediate results. Bridging this gap with a single approach presents substantial challenges, motivating the introduction of CENTAUR, a groundbreaking PPTI framework that seamlessly integrates random permutations and SMPC to address the "impossible trinity". By designing efficient PPTI algorithms tailored to the structural properties of Transformer models, CENTAUR achieves an unprecedented balance among privacy, efficiency, and performance. Our experiments demonstrate CENTAUR's ability to resist diverse data reconstruction attacks, achieve plaintext-level inference accuracy, and boost inference speed by 5.0-30.4 times, unlocking new possibilities for secure and efficient AI deployment.

Related papers

• LAPA-based Dynamic Privacy Optimization for Wireless Federated Learning in Heterogeneous Environments [24.88312051228607]
  Federated Learning (FL) is a distributed machine learning paradigm based on protecting data privacy of devices.<n> Differential privacy (DP) technology reduces the risk of private data leakage by adding artificial noise to the gradients.<n>This paper proposes a Lightweight Adaptive Privacy Allocation (LAPA) strategy, which assigns personalized privacy budgets to devices in each aggregation round.
  arXiv  Detail & Related papers  (2025-05-26T11:00:31Z)
• Private Transformer Inference in MLaaS: A Survey [17.38462391595219]
  Private Transformer Inference (PTI) offers a solution by utilizing cryptographic techniques.<n>This paper reviews recent PTI advancements, highlighting state-of-the-art solutions and challenges.
  arXiv  Detail & Related papers  (2025-05-15T14:00:19Z)
• Federated Learning with Differential Privacy: An Utility-Enhanced Approach [12.614480013684759]
  Federated learning has emerged as an attractive approach to protect data privacy by eliminating the need for sharing clients' data. Recent studies have shown that federated learning alone does not guarantee privacy, as private data may still be inferred from the uploaded parameters to the central server. We present a modification to these vanilla differentially private algorithms based on a Haar wavelet transformation step and a novel noise injection scheme that significantly lowers the bound of the noise variance.
  arXiv  Detail & Related papers  (2025-03-27T04:48:29Z)
• Fed-AugMix: Balancing Privacy and Utility via Data Augmentation [15.325493418326117]
  Gradient leakage attacks pose a significant threat to the privacy guarantees of federated learning.<n>We propose a novel data augmentation-based framework designed to achieve a favorable privacy-utility trade-off.<n>Our framework incorporates the AugMix algorithm at the client level, enabling data augmentation with controllable severity.
  arXiv  Detail & Related papers  (2024-12-18T13:05:55Z)
• A Survey on Private Transformer Inference [17.38462391595219]
  Transformer models have revolutionized AI, enabling applications like content generation and sentiment analysis.<n>However, their use in Machine Learning as a Service (ML) raises significant privacy concerns.<n>Private Transformer Inference (PTI) addresses these issues using cryptographic techniques.
  arXiv  Detail & Related papers  (2024-12-11T07:05:24Z)
• Lightweight Federated Learning with Differential Privacy and Straggler Resilience [19.94124499453864]
  Federated learning (FL) enables collaborative model training through model parameter exchanges instead of raw data.<n>To avoid potential inference attacks from exchanged parameters, differential privacy (DP) offers rigorous guarantee against various attacks.<n>We propose LightDP-FL, a novel lightweight scheme that ensures provable DP against peers and server.
  arXiv  Detail & Related papers  (2024-12-09T00:54:00Z)
• EVA-S3PC: Efficient, Verifiable, Accurate Secure Matrix Multiplication Protocol Assembly and Its Application in Regression [6.706306851710546]
  EVA-S3PC achieves up to 14 significant decimal digits of precision in Float64 calculations. 3-party regression models trained using EVA-S3PC on vertically partitioned data achieve accuracy nearly identical to plaintext training.
  arXiv  Detail & Related papers  (2024-11-05T18:38:44Z)
• Differentially Private Fine-Tuning of Diffusion Models [22.454127503937883]
  The integration of Differential Privacy with diffusion models (DMs) presents a promising yet challenging frontier. Recent developments in this field have highlighted the potential for generating high-quality synthetic data by pre-training on public data. We propose a strategy optimized for private diffusion models, which minimizes the number of trainable parameters to enhance the privacy-utility trade-off.
  arXiv  Detail & Related papers  (2024-06-03T14:18:04Z)
• FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning [54.26614091429253]
  Federated instruction tuning (FedIT) is a promising solution, by consolidating collaborative training across multiple data owners. FedIT encounters limitations such as scarcity of instructional data and risk of exposure to training data extraction attacks. We propose FewFedPIT, designed to simultaneously enhance privacy protection and model performance of federated few-shot learning.
  arXiv  Detail & Related papers  (2024-03-10T08:41:22Z)
• TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data [50.797729676285876]
  We propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm.
  arXiv  Detail & Related papers  (2024-02-16T16:41:14Z)
• SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC [34.63351580241698]
  We introduce a comprehensive PPI framework called SecFormer to achieve fast and accurate PPI for Transformer models.<n>In terms of efficiency, SecFormer is 3.57 and 3.58 times faster than PUMA for BERT$_textBASE$ and BERT$_textLARGE$, demonstrating its effectiveness and speed.
  arXiv  Detail & Related papers  (2024-01-01T15:40:35Z)
• InferDPT: Privacy-Preserving Inference for Black-box Large Language Model [66.07752875835506]
  InferDPT is the first practical framework for the privacy-preserving Inference of black-box LLMs.<n>RANTEXT is a novel differential privacy mechanism integrated into the perturbation module of InferDPT.
  arXiv  Detail & Related papers  (2023-10-18T18:00:11Z)
• East: Efficient and Accurate Secure Transformer Framework for Inference [7.887332345182056]
  We propose a framework emphEast to enable efficient and accurate secure Transformer inference. Compared to Iron, we achieve about 1.8$times$ lower communication within 1.2$times$ lower runtime.
  arXiv  Detail & Related papers  (2023-08-19T06:26:14Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption [112.02441503951297]
  Privacy-preserving inference of transformer models is on the demand of cloud service users. We introduce $textitTHE-X$, an approximation approach for transformers, which enables privacy-preserving inference of pre-trained models.
  arXiv  Detail & Related papers  (2022-06-01T03:49:18Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization [77.43075255745389]
  Federated learning (FL) is capable of preserving private data from mobile terminals (MTs) while training the data into useful models. From a viewpoint of information theory, it is still possible for a curious server to infer private information from the shared models uploaded by MTs. We propose a user-level differential privacy (UDP) algorithm by adding artificial noise to the shared models before uploading them to servers.
  arXiv  Detail & Related papers  (2020-02-29T10:13:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.