Efficient Skip Connections Realization for Secure Inference on Encrypted Data

• URL: http://arxiv.org/abs/2306.06736v1
• Date: Sun, 11 Jun 2023 18:06:06 GMT
• Title: Efficient Skip Connections Realization for Secure Inference on Encrypted Data
• Authors: Nir Drucker and Itamar Zimerman
• Abstract summary: Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. We show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions.
• Score: 3.2996723916635267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving x1.3 computing power improvement for the same accuracy.

Related papers

• A Note on Efficient Privacy-Preserving Similarity Search for Encrypted Vectors [1.3824176915623292]
  Traditional approaches to vector similarity search over encrypted data rely on fully homomorphic encryption (FHE) to enable computation without decryption. This work explores a more efficient alternative: using additively homomorphic encryption (AHE) for privacy-preserving similarity search. We present an efficient algorithm for encrypted similarity search under AHE and analyze its error growth and security implications.
  arXiv  Detail & Related papers  (2025-02-20T06:07:04Z)
• EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation [8.757777529568383]
  Current RAG systems often struggle when retrieval models fail to rank the most relevant documents. We introduce EXIT, an extractive context compression framework. Our evaluations show that EXIT consistently surpasses existing compression methods.
  arXiv  Detail & Related papers  (2024-12-17T05:38:27Z)
• QuanCrypt-FL: Quantized Homomorphic Encryption with Pruning for Secure Federated Learning [0.48342038441006796]
  We propose QuanCrypt-FL, a novel algorithm that combines low-bit quantization and pruning techniques to enhance protection against attacks. We validate our approach on MNIST, CIFAR-10, and CIFAR-100 datasets, demonstrating superior performance compared to state-of-the-art methods. QuanCrypt-FL achieves up to 9x faster encryption, 16x faster decryption, and 1.5x faster inference compared to BatchCrypt, with training time reduced by up to 3x.
  arXiv  Detail & Related papers  (2024-11-08T01:46:00Z)
• Scalable Federated Unlearning via Isolated and Coded Sharding [76.12847512410767]
  Federated unlearning has emerged as a promising paradigm to erase the client-level data effect. This paper proposes a scalable federated unlearning framework based on isolated sharding and coded computing.
  arXiv  Detail & Related papers  (2024-01-29T08:41:45Z)
• SOCI^+: An Enhanced Toolkit for Secure OutsourcedComputation on Integers [50.608828039206365]
  We propose SOCI+ which significantly improves the performance of SOCI. SOCI+ employs a novel (2, 2)-threshold Paillier cryptosystem with fast encryption and decryption as its cryptographic primitive. Compared with SOCI, our experimental evaluation shows that SOCI+ is up to 5.4 times more efficient in computation and 40% less in communication overhead.
  arXiv  Detail & Related papers  (2023-09-27T05:19:32Z)
• Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning [55.198301429316125]
  Machine learning (ML) is a key technique for big-data-driven modelling and analysis of massive Internet of Things (IoT) based intelligent and ubiquitous computing. For fast-increasing applications and data amounts, distributed learning is a promising emerging paradigm since it is often impractical or inefficient to share/aggregate data. This paper studies the problem of training an ML model over decentralized systems, where data are distributed over many user devices.
  arXiv  Detail & Related papers  (2022-02-07T15:04:15Z)
• Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching [72.50506500576746]
  We propose a novel caching paradigm, that we named approximate-key caching. While approximate cache hits alleviate DL inference workload and increase the system throughput, they however introduce an approximation error. We analytically model our caching system performance for classic LRU and ideal caches, we perform a trace-driven evaluation of the expected performance, and we compare the benefits of our proposed approach with the state-of-the-art similarity caching.
  arXiv  Detail & Related papers  (2021-12-13T13:49:11Z)
• Efficient Encrypted Inference on Ensembles of Decision Trees [21.570003967858355]
  Data privacy concerns often prevent the use of cloud-based machine learning services for sensitive personal data. We propose a framework to transfer knowledge extracted by complex decision tree ensembles to shallow neural networks. Our system is highly scalable and can perform efficient inference on batched encrypted (134 bits of security) data with amortized time in milliseconds.
  arXiv  Detail & Related papers  (2021-03-05T01:06:30Z)
• Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search [65.51181219410763]
  One-step methods have been developed to handle pedestrian detection and identification sub-tasks using a single network. There are two major challenges in the current one-step approaches. We propose a decoupled and memory-reinforced network (DMRNet) to overcome these problems.
  arXiv  Detail & Related papers  (2021-02-22T06:19:45Z)
• Efficient CNN Building Blocks for Encrypted Data [6.955451042536852]
  Homomorphic Encryption (FHE) is a promising technique to enable machine learning and inferencing. We show that operational parameters of the chosen FHE scheme have a major impact on the design of the machine learning model. Our empirical study shows that choice of aforementioned design parameters result in significant trade-offs between accuracy, security level, and computational time.
  arXiv  Detail & Related papers  (2021-01-30T21:47:23Z)
• Faster Secure Data Mining via Distributed Homomorphic Encryption [108.77460689459247]
  Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field. We propose a novel general distributed HE-based data mining framework towards one step of solving the scaling problem. We verify the efficiency and effectiveness of our new framework by testing over various data mining algorithms and benchmark data-sets.
  arXiv  Detail & Related papers  (2020-06-17T18:14:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.