SOTERIA: In Search of Efficient Neural Networks for Private Inference

• URL: http://arxiv.org/abs/2007.12934v1
• Date: Sat, 25 Jul 2020 13:53:02 GMT
• Title: SOTERIA: In Search of Efficient Neural Networks for Private Inference
• Authors: Anshul Aggarwal, Trevor E. Carlson, Reza Shokri, Shruti Tople
• Abstract summary: ML-as-a-service is gaining popularity where a cloud server hosts a trained model and offers prediction (inference) service to users. In this setting, our objective is to protect the confidentiality of both the users' input queries as well as the model parameters at the server. We propose SOTERIA, a training method to construct model architectures that are by-design efficient for private inference.
• Score: 15.731520890265545
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ML-as-a-service is gaining popularity where a cloud server hosts a trained model and offers prediction (inference) service to users. In this setting, our objective is to protect the confidentiality of both the users' input queries as well as the model parameters at the server, with modest computation and communication overhead. Prior solutions primarily propose fine-tuning cryptographic methods to make them efficient for known fixed model architectures. The drawback with this line of approach is that the model itself is never designed to operate with existing efficient cryptographic computations. We observe that the network architecture, internal functions, and parameters of a model, which are all chosen during training, significantly influence the computation and communication overhead of a cryptographic method, during inference. Based on this observation, we propose SOTERIA -- a training method to construct model architectures that are by-design efficient for private inference. We use neural architecture search algorithms with the dual objective of optimizing the accuracy of the model and the overhead of using cryptographic primitives for secure inference. Given the flexibility of modifying a model during training, we find accurate models that are also efficient for private computation. We select garbled circuits as our underlying cryptographic primitive, due to their expressiveness and efficiency, but this approach can be extended to hybrid multi-party computation settings. We empirically evaluate SOTERIA on MNIST and CIFAR10 datasets, to compare with the prior work. Our results confirm that SOTERIA is indeed effective in balancing performance and accuracy.

Related papers

• TRAK: Attributing Model Behavior at Scale [79.56020040993947]
  We present TRAK (Tracing with Randomly-trained After Kernel), a data attribution method that is both effective and computationally tractable for large-scale, differenti models.
  arXiv  Detail & Related papers  (2023-03-24T17:56:22Z)
• Evaluating Representations with Readout Model Switching [19.907607374144167]
  In this paper, we propose to use the Minimum Description Length (MDL) principle to devise an evaluation metric. We design a hybrid discrete and continuous-valued model space for the readout models and employ a switching strategy to combine their predictions. The proposed metric can be efficiently computed with an online method and we present results for pre-trained vision encoders of various architectures.
  arXiv  Detail & Related papers  (2023-02-19T14:08:01Z)
• Analyzing the Performance of Deep Encoder-Decoder Networks as Surrogates for a Diffusion Equation [0.0]
  We study the use of encoder-decoder convolutional neural network (CNN) as surrogates for steady-state diffusion solvers. Our results indicate that increasing the size of the training set has a substantial effect on reducing performance fluctuations and overall error.
  arXiv  Detail & Related papers  (2023-02-07T22:53:19Z)
• Towards a learning-based performance modeling for accelerating Deep Neural Networks [1.1549572298362785]
  We start an investigation of predictive models based on machine learning techniques in order to optimize Convolution Neural Networks (CNNs) Preliminary experiments on Midgard-based ARM Mali GPU show that our predictive model outperforms all the convolution operators manually selected by the library.
  arXiv  Detail & Related papers  (2022-12-09T18:28:07Z)
• Prompt Tuning for Parameter-efficient Medical Image Segmentation [79.09285179181225]
  We propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. We pre-train this architecture with a dedicated dense self-supervision scheme based on assignments to online generated prototypes. We demonstrate that the resulting neural network model is able to attenuate the gap between fully fine-tuned and parameter-efficiently adapted models.
  arXiv  Detail & Related papers  (2022-11-16T21:55:05Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Automated Algorithm Selection: from Feature-Based to Feature-Free Approaches [0.5801044612920815]
  We propose a novel technique for algorithm-selection, applicable to optimisation in which there is implicit sequential information encapsulated in the data. We train two types of recurrent neural networks to predict a packing in online bin-packing, selecting from four well-known domains.
  arXiv  Detail & Related papers  (2022-03-24T23:59:50Z)
• Conservative Objective Models for Effective Offline Model-Based Optimization [78.19085445065845]
  Computational design problems arise in a number of settings, from synthetic biology to computer architectures. We propose a method that learns a model of the objective function that lower bounds the actual value of the ground-truth objective on out-of-distribution inputs. COMs are simple to implement and outperform a number of existing methods on a wide range of MBO problems.
  arXiv  Detail & Related papers  (2021-07-14T17:55:28Z)
• GECKO: Reconciling Privacy, Accuracy and Efficiency in Embedded Deep Learning [5.092028049119383]
  We analyse the three-dimensional privacy-accuracy-efficiency tradeoff in NNs for IoT devices. We propose Gecko training methodology where we explicitly add resistance to private inferences as a design objective.
  arXiv  Detail & Related papers  (2020-10-02T10:36:55Z)
• Pre-Trained Models for Heterogeneous Information Networks [57.78194356302626]
  We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network. PF-HIN consistently and significantly outperforms state-of-the-art alternatives on each of these tasks, on four datasets.
  arXiv  Detail & Related papers  (2020-07-07T03:36:28Z)
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data [59.50904660420082]
  Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model.
  arXiv  Detail & Related papers  (2020-05-22T23:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.