Homomorphic WiSARDs: Efficient Weightless Neural Network training over encrypted data

• URL: http://arxiv.org/abs/2403.20190v1
• Date: Fri, 29 Mar 2024 14:09:59 GMT
• Title: Homomorphic WiSARDs: Efficient Weightless Neural Network training over encrypted data
• Abstract summary: Homomorphic evaluation of inference algorithms is practical even for relatively deep Convolution Neural Networks (CNNs) This work introduces the homomorphic evaluation of Wilkie, Stonham, and Aleksander's Recognition Device (WiSARD) and subsequent Weightless Neural Networks (WNNs) for training and inference on encrypted data.
• Score: 3.3748750222488657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread application of machine learning algorithms is a matter of increasing concern for the data privacy research community, and many have sought to develop privacy-preserving techniques for it. Among existing approaches, the homomorphic evaluation of ML algorithms stands out by performing operations directly over encrypted data, enabling strong guarantees of confidentiality. The homomorphic evaluation of inference algorithms is practical even for relatively deep Convolution Neural Networks (CNNs). However, training is still a major challenge, with current solutions often resorting to lightweight algorithms that can be unfit for solving more complex problems, such as image recognition. This work introduces the homomorphic evaluation of Wilkie, Stonham, and Aleksander's Recognition Device (WiSARD) and subsequent Weightless Neural Networks (WNNs) for training and inference on encrypted data. Compared to CNNs, WNNs offer better performance with a relatively small accuracy drop. We develop a complete framework for it, including several building blocks that can be of independent interest. Our framework achieves 91.7% accuracy on the MNIST dataset after only 3.5 minutes of encrypted training (multi-threaded), going up to 93.8% in 3.5 hours. For the HAM10000 dataset, we achieve 67.9% accuracy in just 1.5 minutes, going up to 69.9% after 1 hour. Compared to the state of the art on the HE evaluation of CNN training, Glyph (Lou et al., NeurIPS 2020), these results represent a speedup of up to 1200 times with an accuracy loss of at most 5.4%. For HAM10000, we even achieved a 0.65% accuracy improvement while being 60 times faster than Glyph. We also provide solutions for small-scale encrypted training. In a single thread on a desktop machine using less than 200MB of memory, we train over 1000 MNIST images in 12 minutes or over the entire Wisconsin Breast Cancer dataset in just 11 seconds.

Related papers

• Do Newer Lightweight CNNs Perform Better Under Resource Constraints? A Controlled Multigenerational Study of Architecture, Initialization, Training Budget, and Efficiency [0.0]
  This study compares nine lightweight CNN model packages across CIFAR-10, CIFAR-100, and Tiny ImageNet.<n>We report top-1 accuracy, macro F1, top-5 accuracy, parameter count, FP32 storage, GMACs, batch-size-1 latency on an NVIDIA L4 and AMD 5 5500U CPU.
  arXiv  Detail & Related papers  (2026-07-02T10:14:07Z)
• Efficient Encrypted Computation in Convolutional Spiking Neural Networks with TFHE [1.3047787405425166]
  Homomorphic Encryption (FHE) is a crucial technology for privacy-preserving computation.<n>FHE struggles with continuous non-polynomial functions, as it operates on discrete integers and supports only addition and multiplication.<n>We introduce FHE-DiCSNN, a framework built on the TFHE scheme, utilizing the discrete nature of SNNs for secure and efficient computations.
  arXiv  Detail & Related papers  (2026-03-25T03:15:59Z)
• PrivSpike: Employing Homomorphic Encryption for Private Inference of Deep Spiking Neural Networks [0.08749675983608168]
  Spiking Neural Networks (SNNs) have emerged as an energy-efficient alternative to conventional deep learning approaches.<n> PRIVSPIKE is a privacy-preserving inference framework for SNNs using the CKKS homomorphic encryption scheme.<n>On a consumer-grade CPU, SNN LeNet-5 models achieved inference times of 28 seconds on MNIST and 212 seconds on Neuromorphic MNIST.
  arXiv  Detail & Related papers  (2025-10-05T02:11:40Z)
• HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption [4.164336621664897]
  HETAL is an efficient Homomorphic Encryption based Transfer Learning algorithm. We propose an encrypted matrix multiplication algorithm, which is 1.8 to 323 times faster than prior methods. Experiments show total training times of 567-3442 seconds, which is less than an hour.
  arXiv  Detail & Related papers  (2024-03-21T03:47:26Z)
• Efficient Privacy-Preserving Convolutional Spiking Neural Networks with FHE [1.437446768735628]
  Homomorphic Encryption (FHE) is a key technology for privacy-preserving computation. FHE has limitations in processing continuous non-polynomial functions. We present a framework called FHE-DiCSNN for homomorphic SNNs. FHE-DiCSNN achieves an accuracy of 97.94% on ciphertexts, with a loss of only 0.53% compared to the original network's accuracy of 98.47%.
  arXiv  Detail & Related papers  (2023-09-16T15:37:18Z)
• One-Shot Learning for Periocular Recognition: Exploring the Effect of Domain Adaptation and Data Bias on Deep Representations [59.17685450892182]
  We investigate the behavior of deep representations in widely used CNN models under extreme data scarcity for One-Shot periocular recognition. We improved state-of-the-art results that made use of networks trained with biometric datasets with millions of images. Traditional algorithms like SIFT can outperform CNNs in situations with limited data.
  arXiv  Detail & Related papers  (2023-07-11T09:10:16Z)
• High-Resolution Convolutional Neural Networks on Homomorphically Encrypted Data via Sharding Ciphertexts [0.08999666725996974]
  We extend methods for evaluating DCNNs on images with larger dimensions and many channels, beyond what can be stored in single ciphertexts. We show how existing DCNN models can be regularized during the training process to further improve efficiency and accuracy. These techniques are applied to homomorphically evaluate a DCNN with high accuracy on the high-resolution ImageNet dataset, achieving $80.2%$ top-1 accuracy.
  arXiv  Detail & Related papers  (2023-06-15T15:16:16Z)
• Pixel Difference Networks for Efficient Edge Detection [71.03915957914532]
  We propose a lightweight yet effective architecture named Pixel Difference Network (PiDiNet) for efficient edge detection. Extensive experiments on BSDS500, NYUD, and Multicue datasets are provided to demonstrate its effectiveness. A faster version of PiDiNet with less than 0.1M parameters can still achieve comparable performance among state of the arts with 200 FPS.
  arXiv  Detail & Related papers  (2021-08-16T10:42:59Z)
• Online Limited Memory Neural-Linear Bandits with Likelihood Matching [53.18698496031658]
  We study neural-linear bandits for solving problems where both exploration and representation learning play an important role. We propose a likelihood matching algorithm that is resilient to catastrophic forgetting and is completely online.
  arXiv  Detail & Related papers  (2021-02-07T14:19:07Z)
• Weight Update Skipping: Reducing Training Time for Artificial Neural Networks [0.30458514384586394]
  We propose a new training methodology for ANNs that exploits the observation of improvement of accuracy shows temporal variations. During such time windows, we keep updating bias which ensures the network still trains and avoids overfitting. Such a training approach virtually achieves the same accuracy with considerably less computational cost, thus lower training time.
  arXiv  Detail & Related papers  (2020-12-05T15:12:10Z)
• Kernel Based Progressive Distillation for Adder Neural Networks [71.731127378807]
  Adder Neural Networks (ANNs) which only contain additions bring us a new way of developing deep neural networks with low energy consumption. There is an accuracy drop when replacing all convolution filters by adder filters. We present a novel method for further improving the performance of ANNs without increasing the trainable parameters.
  arXiv  Detail & Related papers  (2020-09-28T03:29:19Z)
• Faster Secure Data Mining via Distributed Homomorphic Encryption [108.77460689459247]
  Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field. We propose a novel general distributed HE-based data mining framework towards one step of solving the scaling problem. We verify the efficiency and effectiveness of our new framework by testing over various data mining algorithms and benchmark data-sets.
  arXiv  Detail & Related papers  (2020-06-17T18:14:30Z)
• Improved Residual Networks for Image and Video Recognition [98.10703825716142]
  Residual networks (ResNets) represent a powerful type of convolutional neural network (CNN) architecture. We show consistent improvements in accuracy and learning convergence over the baseline. Our proposed approach allows us to train extremely deep networks, while the baseline shows severe optimization issues.
  arXiv  Detail & Related papers  (2020-04-10T11:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.