General Techniques for Reducing Key-Switching Overhead in Privacy-Preserving Two-Party Transformer Inference

• URL: http://arxiv.org/abs/2606.25349v1
• Date: Wed, 24 Jun 2026 03:33:17 GMT
• Title: General Techniques for Reducing Key-Switching Overhead in Privacy-Preserving Two-Party Transformer Inference
• Abstract summary: In secure two-party Transformer inference, linear layers are evaluated using FHE through ciphertext or ciphertext-ciphertext matrix multiplications.<n>Existing optimizations rely heavily on packing-specific algorithms, limiting their general applicability.<n>We propose a preprocessing-assisted method for secure attention computation.
• Score: 2.750026875486172
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In secure two-party Transformer inference, linear layers are typically evaluated using Fully Homomorphic Encryption (FHE) through plaintext-ciphertext or ciphertext-ciphertext matrix multiplications, where key switching primarily occurs and dominates computational overhead in both FHE-based and hybrid FHE-MPC systems. Existing optimizations rely heavily on packing-specific algorithms, limiting their general applicability. Targeting this overhead from a packing-independent perspective, we propose a preprocessing-assisted method for secure attention computation. By decomposing attention into precomputable operations and online interactions, this method reduces online inference-phase key switching without modifying existing packing strategies. However, the first method shifting key switching offline introduces additional storage requirements. To address this, we propose storage-communication trade-off techniques that replace large precomputed ciphertexts with modest online communication, enabling flexible deployment under varying resource constraints. While ciphertext-ciphertext matrix multiplication is offloaded to the preprocessing phase in hybrid schemes and the first layer of FHE-based schemes, these operations still persist in the offline stage and subsequent FHE layers. To further optimize it, we propose a fused key-switch technique targeting the multiplication-followed-by-rotation pattern, which frequently arises in existing RNS-CKKS matrix multiplication schemes. By combining relinearization and rotation into a single procedure, this technique reduces the associated computation costs. Analytical evaluations demonstrate that our proposed techniques significantly reduce online key-switch overhead and provide flexible trade-offs between storage and communication without requiring modifications to existing packing strategies.

Related papers

• Revisiting Decentralized Online Convex Optimization with Compressed Communication [68.72667170947358]
  We propose two FTRL-type algorithms for D-OCO with compressed communication.<n>Our first algorithm considers the full-information setting, and can match the existing regret bounds.<n>Our second algorithm is designed for the bandit setting, and can significantly improve both the regret bounds and communication costs.
  arXiv  Detail & Related papers  (2026-07-02T03:45:41Z)
• Privacy-Preserving High-Resolution Image Gradient Computation Based on Fully Homomorphic Encryption [1.342943588774946]
  homomorphic encryption (HE) has emerged as a core method for privacy-preserving image processing.<n>We propose a multi-text privacy-preserving framework for large images, enabling efficient image encryption and computation under the semi-honest model.
  arXiv  Detail & Related papers  (2026-06-02T11:32:15Z)
• Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping [61.459927600301654]
  Multi-condition control is bottlenecked by the conventional concatenate-and-attend'' strategy.<n>Our analysis reveals that much of this cross-modal interaction is spatially or semantically redundant.<n>We propose Position-aligned and Keyword-scoped Attention (PKA), a highly efficient framework designed to eliminate these redundancies.
  arXiv  Detail & Related papers  (2026-02-06T16:39:10Z)
• Volley Revolver: A Novel Matrix-Encoding Method for Privacy-Preserving Deep Learning (Inference++) [0.0]
  Homomorphic encryption has emerged as a promising approach for enabling secure machine learning in untrusted environments.<n>In this paper, we propose an improved encoding and computation framework that removes the requirement that a single encrypted ciphertext must fully contain one input image.<n>Our method reformulates the data layout and homomorphic operations to partition high-resolution inputs across multiple ciphertexts.
  arXiv  Detail & Related papers  (2025-12-21T08:40:31Z)
• Generative Feature Imputing - A Technique for Error-resilient Semantic Communication [46.46641562787869]
  This paper proposes a novel framework, termed generative feature imputing, which comprises three key techniques.<n>First, we introduce a spatial error concentration packetization strategy that spatially concentrates feature distortions by encoding feature elements based on their channel mappings.<n>Second, we propose a generative feature imputing method that utilizes a diffusion model to efficiently reconstruct missing features caused by packet losses.<n>Third, we develop a semantic-aware power allocation scheme that enables unequal error protection by allocating transmission power according to the semantic importance of each packet.
  arXiv  Detail & Related papers  (2025-08-25T12:19:48Z)
• Joint Transmit and Pinching Beamforming for Pinching Antenna Systems (PASS): Optimization-Based or Learning-Based? [89.05848771674773]
  A novel antenna system ()-enabled downlink multi-user multiple-input single-output (MISO) framework is proposed.<n>It consists of multiple waveguides, which equip numerous low-cost antennas, named (PAs)<n>The positions of PAs can be reconfigured to both spanning large-scale path and space.
  arXiv  Detail & Related papers  (2025-02-12T18:54:10Z)
• HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization [55.972018549438964]
  Federated fine-tuning of pre-trained Large Language Models (LLMs) enables task-specific adaptation across diverse datasets while preserving privacy.<n>We propose HAFLQ (Heterogeneous Adaptive Federated Low-Rank Adaptation Fine-tuned LLM with Quantization), a novel framework for efficient and scalable fine-tuning of LLMs in heterogeneous environments.<n> Experimental results on the text classification task demonstrate that HAFLQ reduces memory usage by 31%, lowers communication cost by 49%, improves accuracy by 50%, and achieves faster convergence compared to the baseline method.
  arXiv  Detail & Related papers  (2024-11-10T19:59:54Z)
• Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy [47.997934291881414]
  Existing mean estimation schemes are usually optimized for $L_infty$ geometry and rely on random rotation or Kashin's representation to adapt to $L$ geometry. We introduce a novel privacy accounting method for the sparsified Gaussian mechanism that incorporates the randomness inherent in sparsification into the DP. Unlike previous approaches, our accounting algorithm directly operates in $L$ geometry, yielding MSEs that fast converge to those of the Gaussian mechanism.
  arXiv  Detail & Related papers  (2024-05-02T03:48:47Z)
• A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free! [72.31332210635524]
  Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. We propose a new randomized first-order method which tackles the communication bottleneck by applying randomized compression operators. We prove that our method can solve the problems without any increase in the number of communications compared to the baseline.
  arXiv  Detail & Related papers  (2020-11-03T13:35:53Z)
• A Practical Layer-Parallel Training Algorithm for Residual Networks [41.267919563145604]
  gradient-based algorithms for training ResNets typically require a forward pass of the input data, followed by back-propagating the objective gradient to update parameters. We propose a novel serial-parallel hybrid training strategy to enable the use of data augmentation, together with downsampling filters to reduce the communication cost.
  arXiv  Detail & Related papers  (2020-09-03T06:03:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.