Binarizing Split Learning for Data Privacy Enhancement and Computation Reduction

• URL: http://arxiv.org/abs/2206.04864v1
• Date: Fri, 10 Jun 2022 04:07:02 GMT
• Title: Binarizing Split Learning for Data Privacy Enhancement and Computation Reduction
• Authors: Ngoc Duy Pham, Alsharif Abuadbba, Yansong Gao, Tran Khoa Phan, Naveen Chilamkurti
• Abstract summary: Split learning (SL) enables data privacy preservation by allowing clients to collaboratively train a deep learning model with the server without sharing raw data. In this study, we propose to binarize the SL local layers for faster computation and reduced memory usage. Our results have demonstrated B-SL models are promising for lightweight IoT/mobile applications with high privacy-preservation requirements.
• Score: 8.40552206158625
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Split learning (SL) enables data privacy preservation by allowing clients to collaboratively train a deep learning model with the server without sharing raw data. However, SL still has limitations such as potential data privacy leakage and high computation at clients. In this study, we propose to binarize the SL local layers for faster computation (up to 17.5 times less forward-propagation time in both training and inference phases on mobile devices) and reduced memory usage (up to 32 times less memory and bandwidth requirements). More importantly, the binarized SL (B-SL) model can reduce privacy leakage from SL smashed data with merely a small degradation in model accuracy. To further enhance the privacy preservation, we also propose two novel approaches: 1) training with additional local leak loss and 2) applying differential privacy, which could be integrated separately or concurrently into the B-SL model. Experimental results with different datasets have affirmed the advantages of the B-SL models compared with several benchmark models. The effectiveness of B-SL models against feature-space hijacking attack (FSHA) is also illustrated. Our results have demonstrated B-SL models are promising for lightweight IoT/mobile applications with high privacy-preservation requirements such as mobile healthcare applications.

Related papers

• Enhancing Accuracy-Privacy Trade-off in Differentially Private Split Learning [2.2676798389997863]
  Split learning (SL) aims to protect user data privacy by distributing deep models between client-server and keeping private data locally. Recently proposed model inversion attacks can recover the original data from the smashed data. A strategy is to adopt differential privacy (DP), which involves safeguarding the smashed data at the expense of some accuracy loss.
  arXiv  Detail & Related papers  (2023-10-22T22:45:13Z)
• Love or Hate? Share or Split? Privacy-Preserving Training Using Split Learning and Homomorphic Encryption [47.86010265348072]
  Split learning (SL) is a new collaborative learning technique that allows participants to train machine learning models without the client sharing raw data. Previous works demonstrated that reconstructing activation maps could result in privacy leakage of client data. In this paper, we improve upon previous works by constructing a protocol based on U-shaped SL that can operate on homomorphically encrypted data.
  arXiv  Detail & Related papers  (2023-09-19T10:56:08Z)
• A More Secure Split: Enhancing the Security of Privacy-Preserving Split Learning [2.853180143237022]
  Split learning (SL) is a new collaborative learning technique that allows participants to train machine learning models without the client sharing raw data. Previous works demonstrated that reconstructing Activation Maps (AMs) could result in privacy leakage of client data. In this paper, we improve upon previous works by constructing a protocol based on U-shaped SL that can operate on homomorphically encrypted data.
  arXiv  Detail & Related papers  (2023-09-15T18:39:30Z)
• Evaluating Privacy Leakage in Split Learning [8.841387955312669]
  On-device machine learning allows us to avoid sharing raw data with a third-party server during inference. Split Learning (SL) is a promising approach that can overcome limitations. In SL, a large machine learning model is divided into two parts, with the bigger part residing on the server side and a smaller part executing on-device.
  arXiv  Detail & Related papers  (2023-05-22T13:00:07Z)
• Split Learning without Local Weight Sharing to Enhance Client-side Data Privacy [11.092451849022268]
  Split learning (SL) aims to protect user data privacy by distributing deep models between client-server and keeping private data locally. This paper first reveals data privacy leakage exacerbated from local weight sharing among the clients in SL through model inversion attacks. We propose and analyze privacy-enhanced SL (P-SL) (or SL without local weight sharing) to reduce the data privacy leakage issue.
  arXiv  Detail & Related papers  (2022-12-01T03:35:14Z)
• Federated Zero-Shot Learning for Visual Recognition [55.65879596326147]
  We propose a novel Federated Zero-Shot Learning FedZSL framework. FedZSL learns a central model from the decentralized data residing on edge devices. The effectiveness and robustness of FedZSL are demonstrated by extensive experiments conducted on three zero-shot benchmark datasets.
  arXiv  Detail & Related papers  (2022-09-05T14:49:34Z)
• OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning [110.40285771431687]
  Semi-supervised learning (SSL) is one of the dominant approaches to address the annotation bottleneck of supervised learning. Recent SSL methods can effectively leverage a large repository of unlabeled data to improve performance while relying on a small set of labeled data. This work introduces OpenLDN that utilizes a pairwise similarity loss to discover novel classes.
  arXiv  Detail & Related papers  (2022-07-05T18:51:05Z)
• Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of Semi-Supervised Learning and Active Learning [60.26659373318915]
  Active learning (AL) and semi-supervised learning (SSL) are two effective, but often isolated, means to alleviate the data-hungry problem. We propose an innovative Inconsistency-based virtual aDvErial algorithm to further investigate SSL-AL's potential superiority. Two real-world case studies visualize the practical industrial value of applying and deploying the proposed data sampling algorithm.
  arXiv  Detail & Related papers  (2022-06-07T13:28:43Z)
• DATA: Domain-Aware and Task-Aware Pre-training [94.62676913928831]
  We present DATA, a simple yet effective NAS approach specialized for self-supervised learning (SSL) Our method achieves promising results across a wide range of computation costs on downstream tasks, including image classification, object detection and semantic segmentation.
  arXiv  Detail & Related papers  (2022-03-17T02:38:49Z)
• Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning [82.06357027523262]
  Federated learning (FL) and split learning (SL) are two spearheads possessing their pros and cons, and are suited for many user clients and large models. In this work, we first identify the fundamental bottlenecks of SL, and thereby propose a scalable SL framework, coined SGLR.
  arXiv  Detail & Related papers  (2021-12-11T08:33:25Z)
• SplitFed: When Federated Learning Meets Split Learning [16.212941272007285]
  Federated learning (FL) and split learning (SL) are two popular distributed machine learning approaches. This paper presents a novel approach, named splitfed learning (SFL), that amalgamates the two approaches. SFL provides similar test accuracy and communication efficiency as SL while significantly decreasing its computation time per global epoch than in SL for multiple clients.
  arXiv  Detail & Related papers  (2020-04-25T08:52:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.