NN-EMD: Efficiently Training Neural Networks using Encrypted Multi-Sourced Datasets

• URL: http://arxiv.org/abs/2012.10547v2
• Date: Sun, 18 Apr 2021 00:49:07 GMT
• Title: NN-EMD: Efficiently Training Neural Networks using Encrypted Multi-Sourced Datasets
• Authors: Runhua Xu, James Joshi and Chao Li
• Abstract summary: Training a machine learning model over an encrypted dataset is an existing promising approach to address the privacy-preserving machine learning task. We propose a novel framework, NN-EMD, to train a deep neural network (DNN) model over multiple datasets collected from multiple sources. We evaluate our framework for performance with regards to the training time and model accuracy on the MNIST datasets.
• Score: 7.067870969078555
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Training a machine learning model over an encrypted dataset is an existing promising approach to address the privacy-preserving machine learning task, however, it is extremely challenging to efficiently train a deep neural network (DNN) model over encrypted data for two reasons: first, it requires large-scale computation over huge datasets; second, the existing solutions for computation over encrypted data, such as homomorphic encryption, is inefficient. Further, for an enhanced performance of a DNN model, we also need to use huge training datasets composed of data from multiple data sources that may not have pre-established trust relationships among each other. We propose a novel framework, NN-EMD, to train DNN over multiple encrypted datasets collected from multiple sources. Toward this, we propose a set of secure computation protocols using hybrid functional encryption schemes. We evaluate our framework for performance with regards to the training time and model accuracy on the MNIST datasets. Compared to other existing frameworks, our proposed NN-EMD framework can significantly reduce the training time, while providing comparable model accuracy and privacy guarantees as well as supporting multiple data sources. Furthermore, the depth and complexity of neural networks do not affect the training time despite introducing a privacy-preserving NN-EMD setting.

Related papers

• Data Filtering Networks [67.827994353269]
  We study the problem of learning a data filtering network (DFN) for this second step of filtering a large uncurated dataset. Our key finding is that the quality of a network for filtering is distinct from its performance on downstream tasks. Based on our insights, we construct new data filtering networks that induce state-of-the-art image-text datasets.
  arXiv  Detail & Related papers  (2023-09-29T17:37:29Z)
• Towards Robust k-Nearest-Neighbor Machine Translation [72.9252395037097]
  k-Nearest-Neighbor Machine Translation (kNN-MT) becomes an important research direction of NMT in recent years. Its main idea is to retrieve useful key-value pairs from an additional datastore to modify translations without updating the NMT model. The underlying retrieved noisy pairs will dramatically deteriorate the model performance. We propose a confidence-enhanced kNN-MT model with robust training to alleviate the impact of noise.
  arXiv  Detail & Related papers  (2022-10-17T07:43:39Z)
• Neural Attentive Circuits [93.95502541529115]
  We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs achieve an 8x speedup at inference time while losing less than 3% performance.
  arXiv  Detail & Related papers  (2022-10-14T18:00:07Z)
• Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design [28.789702559193675]
  We propose SPNN - a Scalable and Privacy-preserving deep Neural Network learning framework. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption. Experimental results conducted on real-world datasets demonstrate the superiority of SPNN.
  arXiv  Detail & Related papers  (2020-12-17T02:26:16Z)
• Deep Time Delay Neural Network for Speech Enhancement with Full Data Learning [60.20150317299749]
  This paper proposes a deep time delay neural network (TDNN) for speech enhancement with full data learning. To make full use of the training data, we propose a full data learning method for speech enhancement.
  arXiv  Detail & Related papers  (2020-11-11T06:32:37Z)
• Optimal training of integer-valued neural networks with mixed integer programming [2.528056693920671]
  We develop new MIP models which improve training efficiency and which can train the important class of integer-valued neural networks (INNs) We provide a batch training method that dramatically increases the amount of data that MIP solvers can use to train. Experimental results on two real-world data-limited datasets demonstrate that our approach strongly outperforms the previous state of the art in training NNs with MIP.
  arXiv  Detail & Related papers  (2020-09-08T15:45:44Z)
• POSEIDON: Privacy-Preserving Federated Neural Network Learning [8.103262600715864]
  POSEIDON is a first of its kind in the regime of privacy-preserving neural network training. It employs multiparty lattice-based cryptography to preserve the confidentiality of the training data, the model, and the evaluation data. It trains a 3-layer neural network on the MNIST dataset with 784 features and 60K samples distributed among 10 parties in less than 2 hours.
  arXiv  Detail & Related papers  (2020-09-01T11:06:31Z)
• Industrial Scale Privacy Preserving Deep Neural Network [23.690146141150407]
  We propose an industrial scale privacy preserving neural network learning paradigm, which is secure against semi-honest adversaries. We conduct experiments on real-world fraud detection dataset and financial distress prediction dataset.
  arXiv  Detail & Related papers  (2020-03-11T10:15:37Z)
• Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment [84.57874289554839]
  Training deep neural networks on large-scale datasets requires significant hardware resources. Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize. We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
  arXiv  Detail & Related papers  (2020-02-10T16:20:02Z)
• CryptoSPN: Privacy-preserving Sum-Product Network Inference [84.88362774693914]
  We present a framework for privacy-preserving inference of sum-product networks (SPNs) CryptoSPN achieves highly efficient and accurate inference in the order of seconds for medium-sized SPNs.
  arXiv  Detail & Related papers  (2020-02-03T14:49:18Z)
• Constructing Deep Neural Networks with a Priori Knowledge of Wireless Tasks [37.060397377445504]
  Two kinds of permutation invariant properties widely existed in wireless tasks can be harnessed to reduce the number of model parameters. We find special architecture of DNNs whose input-output relationships satisfy the properties, called permutation invariant DNN (PINN) We take predictive resource allocation and interference coordination as examples to show how the PINNs can be employed for learning the optimal policy with unsupervised and supervised learning.
  arXiv  Detail & Related papers  (2020-01-29T08:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.