HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption

• URL: http://arxiv.org/abs/2403.14111v1
• Date: Thu, 21 Mar 2024 03:47:26 GMT
• Title: HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption
• Authors: Seewoo Lee, Garam Lee, Jung Woo Kim, Junbum Shin, Mun-Kyu Lee,
• Abstract summary: HETAL is an efficient Homomorphic Encryption based Transfer Learning algorithm. We propose an encrypted matrix multiplication algorithm, which is 1.8 to 323 times faster than prior methods. Experiments show total training times of 567-3442 seconds, which is less than an hour.
• Score: 4.164336621664897
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transfer learning is a de facto standard method for efficiently training machine learning models for data-scarce problems by adding and fine-tuning new classification layers to a model pre-trained on large datasets. Although numerous previous studies proposed to use homomorphic encryption to resolve the data privacy issue in transfer learning in the machine learning as a service setting, most of them only focused on encrypted inference. In this study, we present HETAL, an efficient Homomorphic Encryption based Transfer Learning algorithm, that protects the client's privacy in training tasks by encrypting the client data using the CKKS homomorphic encryption scheme. HETAL is the first practical scheme that strictly provides encrypted training, adopting validation-based early stopping and achieving the accuracy of nonencrypted training. We propose an efficient encrypted matrix multiplication algorithm, which is 1.8 to 323 times faster than prior methods, and a highly precise softmax approximation algorithm with increased coverage. The experimental results for five well-known benchmark datasets show total training times of 567-3442 seconds, which is less than an hour.

Related papers

• Robust Representation Learning for Privacy-Preserving Machine Learning: A Multi-Objective Autoencoder Approach [0.9831489366502302]
  We propose a robust representation learning framework for privacy-preserving machine learning (ppML) Our method centers on training autoencoders in a multi-objective manner and then concatenating the latent and learned features from the encoding part as the encoded form of our data. With our proposed framework, we can share our data and use third party tools without being under the threat of revealing its original form.
  arXiv  Detail & Related papers  (2023-09-08T16:41:25Z)
• When approximate design for fast homomorphic computation provides differential privacy guarantees [0.08399688944263842]
  Differential privacy (DP) and cryptographic primitives are popular countermeasures against privacy attacks. In this paper, we design SHIELD, a probabilistic approximation algorithm for the argmax operator. Even if SHIELD could have other applications, we here focus on one setting and seamlessly integrate it in the SPEED collaborative training framework.
  arXiv  Detail & Related papers  (2023-04-06T09:38:01Z)
• PEOPL: Characterizing Privately Encoded Open Datasets with Public Labels [59.66777287810985]
  We introduce information-theoretic scores for privacy and utility, which quantify the average performance of an unfaithful user. We then theoretically characterize primitives in building families of encoding schemes that motivate the use of random deep neural networks.
  arXiv  Detail & Related papers  (2023-03-31T18:03:53Z)
• Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning [63.45532264721498]
  Self-supervised learning is an emerging technique to pre-train encoders using unlabeled data. We perform first systematic, principled measurement study to understand whether and when a pretrained encoder can address the limitations of secure or privacy-preserving supervised learning algorithms.
  arXiv  Detail & Related papers  (2022-12-06T21:35:35Z)
• Application of Data Encryption in Chinese Named Entity Recognition [11.084360853065736]
  We propose an encryption learning framework to address the problems of data leakage and inconvenient disclosure of sensitive data. We introduce multiple encryption algorithms to encrypt training data in the named entity recognition task for the first time. The experimental results show that the encryption method achieves satisfactory results.
  arXiv  Detail & Related papers  (2022-08-31T04:20:37Z)
• Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
  Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples. We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment. We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
  arXiv  Detail & Related papers  (2022-03-23T06:24:31Z)
• On Deep Learning with Label Differential Privacy [54.45348348861426]
  We study the multi-class classification setting where the labels are considered sensitive and ought to be protected. We propose a new algorithm for training deep neural networks with label differential privacy, and run evaluations on several datasets.
  arXiv  Detail & Related papers  (2021-02-11T15:09:06Z)
• Faster Secure Data Mining via Distributed Homomorphic Encryption [108.77460689459247]
  Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field. We propose a novel general distributed HE-based data mining framework towards one step of solving the scaling problem. We verify the efficiency and effectiveness of our new framework by testing over various data mining algorithms and benchmark data-sets.
  arXiv  Detail & Related papers  (2020-06-17T18:14:30Z)
• Cryptotree: fast and accurate predictions on encrypted structured data [0.0]
  Homomorphic Encryption (HE) is acknowledged for its ability to allow computation on encrypted data, where both the input and output are encrypted. We propose Cryptotree, a framework that enables the use of Random Forests (RF), a very powerful learning procedure compared to linear regression.
  arXiv  Detail & Related papers  (2020-06-15T11:48:01Z)
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization [77.43075255745389]
  Federated learning (FL) is capable of preserving private data from mobile terminals (MTs) while training the data into useful models. From a viewpoint of information theory, it is still possible for a curious server to infer private information from the shared models uploaded by MTs. We propose a user-level differential privacy (UDP) algorithm by adding artificial noise to the shared models before uploading them to servers.
  arXiv  Detail & Related papers  (2020-02-29T10:13:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.