MPC-Pipe: an Efficient Pipeline Scheme for Secure Multi-party Machine Learning Inference

• URL: http://arxiv.org/abs/2209.13643v2
• Date: Tue, 27 Aug 2024 17:32:39 GMT
• Title: MPC-Pipe: an Efficient Pipeline Scheme for Secure Multi-party Machine Learning Inference
• Authors: Yongqin Wang, Rachit Rajat, Murali Annavaram,
• Abstract summary: We show that it is possible to carefully orchestrate the computation and communication steps to overlap. We propose MPC-Pipe, an efficient MPC system for both training and inference of ML workloads.
• Score: 5.7203077366666015
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-party computing (MPC) has been gaining popularity as a secure computing model over the past few years. However, prior works have demonstrated that MPC protocols still pay substantial performance penalties compared to plaintext, particularly when applied to ML algorithms. The overhead is due to added computation and communication costs. Prior studies, as well as our own analysis, found that most MPC protocols today sequentially perform communication and computation. The participating parties must compute on their shares first and then perform data communication to allow the distribution of new secret shares before proceeding to the next computation step. In this work, we show that serialization is unnecessary, particularly in the context of ML computations (both in Convolutional neural networks and in Transformer-based models). We demonstrate that it is possible to carefully orchestrate the computation and communication steps to overlap. We propose MPC-Pipe, an efficient MPC system for both training and inference of ML workloads, which pipelines computations and communications in an MPC protocol during the online phase. MPC-Pipe proposes three pipeline schemes to optimize the online phase of ML in the semi-honest majority adversary setting. We implement MPC-Pipe by augmenting a modified version of CrypTen, which separates online and offline phases. We evaluate the end-to-end system performance benefits of the online phase of MPC using deep neural networks (VGG16, ResNet50) and Transformers using different network settings. We show that MPC-Pipe can improve the throughput and latency of ML workloads.

Related papers

• Low-Latency Privacy-Preserving Deep Learning Design via Secure MPC [31.35072624488929]
  Secure multi-party computation (MPC) facilitates privacy-preserving computation between multiple parties without leaking private information. This work proposes a low-latency secret-sharing-based MPC design that reduces unnecessary communication rounds during the execution of MPC protocols.
  arXiv  Detail & Related papers  (2024-07-24T07:01:21Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• On Building Myopic MPC Policies using Supervised Learning [0.0]
  This paper considers an alternative strategy, where supervised learning is used to learn the optimal value function offline instead of learning the optimal policy. This can then be used as the cost-to-go function in a myopic MPC with a very short prediction horizon.
  arXiv  Detail & Related papers  (2024-01-23T08:08:09Z)
• Fast and Private Inference of Deep Neural Networks by Co-designing Activation Functions [26.125340303868335]
  Current approaches suffer from large inference times. We propose a novel training algorithm that gives accuracy competitive with inferences models. Our evaluation shows between $3$ and $110times$ speedups in inference time on large models with up to $23$ million parameters.
  arXiv  Detail & Related papers  (2023-06-14T14:38:25Z)
• Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis [59.127630388320036]
  A wireless federated learning system is investigated by allowing a server and workers to exchange uncoded information via wireless channels. Since the workers frequently upload local to the server via bandwidth-limited channels, the uplink transmission from the workers to the server becomes a communication bottleneck. A one-shot distributed principle component analysis (PCA) is leveraged to reduce the dimension of the dimension of the communication bottleneck.
  arXiv  Detail & Related papers  (2023-03-31T08:41:42Z)
• Partitioning Distributed Compute Jobs with Reinforcement Learning and Graph Neural Networks [58.720142291102135]
  Large-scale machine learning models are bringing advances to a broad range of fields. Many of these models are too large to be trained on a single machine, and must be distributed across multiple devices. We show that maximum parallelisation is sub-optimal in relation to user-critical metrics such as throughput and blocking rate.
  arXiv  Detail & Related papers  (2023-01-31T17:41:07Z)
• Fair and Efficient Distributed Edge Learning with Hybrid Multipath TCP [62.81300791178381]
  The bottleneck of distributed edge learning over wireless has shifted from computing to communication. Existing TCP-based data networking schemes for DEL are application-agnostic and fail to deliver adjustments according to application layer requirements. We develop a hybrid multipath TCP (MP TCP) by combining model-based and deep reinforcement learning (DRL) based MP TCP for DEL.
  arXiv  Detail & Related papers  (2022-11-03T09:08:30Z)
• Towards Semantic Communication Protocols: A Probabilistic Logic Perspective [69.68769942563812]
  We propose a semantic protocol model (SPM) constructed by transforming an NPM into an interpretable symbolic graph written in the probabilistic logic programming language (ProbLog) By leveraging its interpretability and memory-efficiency, we demonstrate several applications such as SPM reconfiguration for collision-avoidance.
  arXiv  Detail & Related papers  (2022-07-08T14:19:36Z)
• HD-cos Networks: Efficient Neural Architectures for Secure Multi-Party Computation [26.67099154998755]
  Multi-party computation (MPC) is a branch of cryptography where multiple non-colluding parties execute a protocol to securely compute a function. We study training and inference of neural networks under the MPC setup. We show that both of the approaches enjoy strong theoretical motivations and efficient computation under the MPC setup.
  arXiv  Detail & Related papers  (2021-10-28T21:15:11Z)
• Adam in Private: Secure and Fast Training of Deep Neural Networks with Adaptive Moment Estimation [6.342794803074475]
  We propose a framework that allows efficient evaluation of full-fledged state-of-the-art machine learning algorithms. This is in contrast to most prior works, which substitute ML algorithms with approximated "MPC-friendly" variants. We obtain secure training that outperforms state-of-the-art three-party systems.
  arXiv  Detail & Related papers  (2021-06-04T01:40:09Z)
• Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits [99.59941892183454]
  We propose Einsum Networks (EiNets), a novel implementation design for PCs. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation. We show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation.
  arXiv  Detail & Related papers  (2020-04-13T23:09:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.