OptiPrime: Optimizing Private Inference through Protocol-Hardware Co-design

• URL: http://arxiv.org/abs/2609.16898v1
• Date: Tue, 15 Sep 2026 09:29:58 GMT
• Title: OptiPrime: Optimizing Private Inference through Protocol-Hardware Co-design
• Abstract summary: Private deep neural network (DNN) inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC) can protect user data with a formal guarantee, but at the cost of significant latency overhead due to HE.<n>We introduce OptiPrime, a protocol- hardware co-optimization framework for efficient private DNN inference.<n>Our framework outperforms the Cheetah baseline by most 5.7 times on CPUs and 4.2 times with an accelerator.
• Score: 11.152255710014694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Private deep neural network (DNN) inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC) can protect user data with a formal guarantee, but at the cost of significant latency overhead due to HE. Customized HE accelerators have been proposed and have achieved orders-of-magnitude speedup for individual HE operations. However, when directly applying a commercial HE accelerator to state-of-the-art HE-MPC frameworks, we observe only limited end-to-end performance gain. This is because HE-MPC frameworks often require wireless transmission of input and output ciphertexts for each HE operation, leading to a severe network communication bottleneck. To overcome this challenge, we introduce OptiPrime, a protocol-hardware co-optimization framework for efficient private DNN inference. OptiPrime features a novel HE protocol for convolutions that substantially reduces the number of transmitted output ciphertexts and mitigates the network communication bottleneck. Meanwhile, as the new protocol introduces complex computation for fewer output ciphertext, we observe new memory access challenges due to a high volume of weight plaintexts and intermediate ciphertexts. Hence, we further propose a lightweight compression system for the weight plaintexts, reducing memory traffic by 10 times, as well as a specialized dataflow to maximize on-chip data reuse of intermediate ciphertexts. Extensive experiments show that our framework outperforms the Cheetah baseline by at most 5.7 times on CPUs and 4.2 times with an accelerator.

Related papers

• Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation [49.97373153077311]
  We investigate the decoding overhead associated with a centroid encoding for Instance, in which each pixel regresses a positional embedding of its instance centroid.<n>Our solution significantly reduces decoding overhead and improves Endto-End latency, outperforming both CPU-based approaches and naive GPU implementations.
  arXiv  Detail & Related papers  (2026-09-15T09:05:17Z)
• Almost-Free Queue Jumping for Prior Inputs in Private Neural Inference [14.074510817815538]
  PP-ML enables secure neural network inference by integrating cryptographic primitives.<n>PrivQJ exploits shared computation across inputs via in-processing slot recycling.<n>Results demonstrate over an order-of-magnitude reduction in overhead compared to state-of-the-art PP-ML systems.
  arXiv  Detail & Related papers  (2026-03-13T12:41:36Z)
• FedBit: Accelerating Privacy-Preserving Federated Learning via Bit-Interleaved Packing and Cross-Layer Co-Design [2.255961793913651]
  Federated learning (FL) with fully homomorphic encryption (FHE) effectively safeguards data privacy during model aggregation.<n>FedBit is a hardware/software co-designed framework for the Brakerski-Fan-Vercauteren (BFV) scheme.<n>FedBit employs bit-interleaved data packing to embed multiple model parameters into a single ciphertext coefficient.
  arXiv  Detail & Related papers  (2025-09-27T03:58:16Z)
• ENSI: Efficient Non-Interactive Secure Inference for Large Language Models [10.82684192498215]
  We propose ENSI, a novel secure inference framework for large language models (LLMs)<n>ENSI employs an optimized encoding strategy that seamlessly integrates CKKS scheme with a lightweight LLM variant, BitNet.<n>We demonstrate that ENSI achieves approximately an 8x acceleration in matrix multiplications and a 2.6x speedup in softmax inference on CPU.
  arXiv  Detail & Related papers  (2025-09-11T13:04:22Z)
• GDNTT: an Area-Efficient Parallel NTT Accelerator Using Glitch-Driven Near-Memory Computing and Reconfigurable 10T SRAM [14.319119105134309]
  This paper proposes an area-efficient highly parallel NTT accelerator with glitch-driven near-memory computing (GDNTT)<n>The design integrates a 10T for data storage, enabling flexible row/column data access and streamlining circuit mapping strategies.<n> Evaluation results show that the proposed NTT accelerator achieves a 1.528* improvement in throughput-per-area compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2025-05-13T01:53:07Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [54.53508601749513]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework.<n>To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features.<n>Results show that TOFC achieves up to 52% reduction in data transmission overhead and 63% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation [8.859237832459876]
  We present FASTLMPI, a new approach to accelerate private TBM inference through fine-grained optimization.<n>Specifically, through the fine-grained co-design of homomorphic encryption and secret sharing, FASTLMPI achieves efficient protocols for matrix multiplication, SoftMax, LayerNorm, and GeLULU.<n>FASTLMPI shows a remarkable 54% to 64% decrease in runtime and an impressive 72.2% reduction in communication costs.
  arXiv  Detail & Related papers  (2024-12-21T08:33:12Z)
• Accelerating Error Correction Code Transformers [56.75773430667148]
  We introduce a novel acceleration method for transformer-based decoders. We achieve a 90% compression ratio and reduce arithmetic operation energy consumption by at least 224 times on modern hardware.
  arXiv  Detail & Related papers  (2024-10-08T11:07:55Z)
• Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR [67.63332492134332]
  We design an optimized conformer that is small enough to meet on-device restrictions and has fast inference on TPUs. Our proposed encoder can double as a strong standalone encoder in on device, and as the first part of a high-performance ASR pipeline.
  arXiv  Detail & Related papers  (2023-03-31T23:30:48Z)
• RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inference [17.299835585861747]
  We introduce RRNet, a framework that aims to jointly reduce the overhead of MPC comparison protocols and accelerate computation through hardware acceleration. Our approach integrates the hardware latency of cryptographic building blocks into the DNN loss function, resulting in improved energy efficiency, accuracy, and security guarantees.
  arXiv  Detail & Related papers  (2023-02-05T04:02:13Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference [82.1584439276834]
  Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks. We present EdgeBERT, an in-depth algorithm- hardware co-design for latency-aware energy optimization for multi-task NLP.
  arXiv  Detail & Related papers  (2020-11-28T19:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.