Split Unlearning

• URL: http://arxiv.org/abs/2308.10422v3
• Date: Fri, 30 Aug 2024 08:46:27 GMT
• Title: Split Unlearning
• Authors: Guangsheng Yu, Yanna Jiang, Qin Wang, Xu Wang, Baihe Ma, Caijun Sun, Wei Ni, Ren Ping Liu,
• Abstract summary: We propose, implement, and evaluate a practical Split Unlearning framework by enabling SISA-based machine unlearning (SP'21) in Split Learning (SL) We introduce SplitWiper and SplitWiper+, which leverage the inherent "Sharded" structure of SL and address the issues where existing SL methods compromise the "Isolated" principle of SISA. Our design decouples the propagation of neural signals between clients and between clients and the server, enabling SISA-based unlearning in SplitWiper, even with absent clients.
• Score: 23.380995944550484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We for the first time propose, implement, and evaluate a practical Split Unlearning framework by enabling SISA-based machine unlearning (SP'21) in Split Learning (SL). We introduce SplitWiper and SplitWiper+, which leverage the inherent "Sharded" structure of SL and address the issues where existing SL methods compromise the "Isolated" principle of SISA due to the tight coupling between clients and the server. This coupling involves frequent bidirectional data flows and iterative training across all clients. We resolve these challenges by designing a new one-way-one-off propagation scheme. Our design decouples the propagation of neural signals between clients and between clients and the server, enabling SISA-based unlearning in SplitWiper, even with absent clients. SplitWiper+ further enhances client label privacy against the server under this scheme by using differential privacy. Our extensive experiments across diverse data distributions and tasks demonstrate that SplitWiper, which involves only the requesting clients, achieves 0% unlearning accuracy and is over 108% more effective in retaining accuracy than existing SL methods, while maintaining constant overhead through its one-way-one-off propagation, reducing computational and communication costs by over 99%. SplitWiper+ preserves over 90% of label privacy when sharing masked labels with servers.

Related papers

• CURE: Privacy-Preserving Split Learning Done Right [1.388112207221632]
  Homomorphic encryption (HE)-based solutions exist for this scenario but often impose prohibitive computational burdens. CURE is a novel system that encrypts only the server side of the model and the data. We demonstrate CURE can achieve similar accuracy to plaintext SL while being 16x more efficient in terms of the runtime.
  arXiv  Detail & Related papers  (2024-07-12T04:10:19Z)
• Federated Learning with Only Positive Labels by Exploring Label Correlations [78.59613150221597]
  Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting. We propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC)
  arXiv  Detail & Related papers  (2024-04-24T02:22:50Z)
• Subspace based Federated Unlearning [75.90552823500633]
  Federated unlearning (FL) aims to remove a specified target client's contribution in FL to satisfy the user's right to be forgotten. Most existing federated unlearning algorithms require the server to store the history of the parameter updates. We propose a simple-yet-effective subspace based federated unlearning method, dubbed SFU, that lets the global model perform gradient ascent.
  arXiv  Detail & Related papers  (2023-02-24T04:29:44Z)
• SplitOut: Out-of-the-Box Training-Hijacking Detection in Split Learning via Outlier Detection [0.0]
  Split learning enables efficient and privacy-aware training of a deep neural network by splitting a neural network so that the clients (data holders) compute the first layers and only share the intermediate output with the central compute-heavy server. Server has full control over what the client models learn, which has already been exploited to infer the private data of clients and to implement backdoors in the client models. We show that given modest assumptions regarding the clients' compute capabilities, an out-of-the-box detection method can be used to detect existing training-hijacking attacks with almost-zero false positive rates.
  arXiv  Detail & Related papers  (2023-02-16T23:02:39Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
  Clustered learning has been shown to produce promising results by grouping clients into clusters. Existing FL algorithms are essentially trying to group clients together with similar distributions. Prior FL algorithms attempt similarities indirectly during training.
  arXiv  Detail & Related papers  (2022-09-21T17:37:54Z)
• Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning [82.06357027523262]
  Federated learning (FL) and split learning (SL) are two spearheads possessing their pros and cons, and are suited for many user clients and large models. In this work, we first identify the fundamental bottlenecks of SL, and thereby propose a scalable SL framework, coined SGLR.
  arXiv  Detail & Related papers  (2021-12-11T08:33:25Z)
• Federated Learning with Taskonomy for Non-IID Data [0.0]
  We introduce federated learning with taskonomy. In a one-off process, the server provides the clients with a pretrained (and fine-tunable) encoder to compress their data into a latent representation, and transmit the signature of their data back to the server. The server then learns the task-relatedness among clients via manifold learning, and performs a generalization of federated averaging.
  arXiv  Detail & Related papers  (2021-03-29T20:47:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.