A few-shot Label Unlearning in Vertical Federated Learning

• URL: http://arxiv.org/abs/2410.10922v1
• Date: Mon, 14 Oct 2024 12:08:12 GMT
• Title: A few-shot Label Unlearning in Vertical Federated Learning
• Authors: Hanlin Gu, Hong Xi Tae, Chee Seng Chan, Lixin Fan,
• Abstract summary: This paper addresses the critical challenge of unlearning in Vertical Federated Learning (VFL) We introduce the first approach specifically designed to tackle label unlearning in VFL, focusing on scenarios where the active party aims to mitigate the risk of label leakage. Our method leverages a limited amount of labeled data, utilizing manifold mixup to augment the forward embedding of insufficient data, followed by gradient ascent on the augmented embeddings to erase label information from the models.
• Score: 16.800865928660954
• License:
• Abstract: This paper addresses the critical challenge of unlearning in Vertical Federated Learning (VFL), an area that has received limited attention compared to horizontal federated learning. We introduce the first approach specifically designed to tackle label unlearning in VFL, focusing on scenarios where the active party aims to mitigate the risk of label leakage. Our method leverages a limited amount of labeled data, utilizing manifold mixup to augment the forward embedding of insufficient data, followed by gradient ascent on the augmented embeddings to erase label information from the models. This combination of augmentation and gradient ascent enables high unlearning effectiveness while maintaining efficiency, completing the unlearning procedure within seconds. Extensive experiments conducted on diverse datasets, including MNIST, CIFAR10, CIFAR100, and ModelNet, validate the efficacy and scalability of our approach. This work represents a significant advancement in federated learning, addressing the unique challenges of unlearning in VFL while preserving both privacy and computational efficiency.

Related papers

• Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios [18.73206066109299]
  Mini-Unlearning is a novel approach that capitalizes on a critical observation: unlearned parameters correlate with retrained parameters through contraction mapping. This lightweight, scalable method significantly enhances model accuracy and strengthens resistance to membership inference attacks. Our experiments demonstrate that Mini-Unlearning not only works under higher unlearning ratios but also outperforms existing techniques in both accuracy and security.
  arXiv  Detail & Related papers  (2024-06-24T01:43:30Z)
• Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classification [48.81069245141415]
  We introduce a novel contrastive loss function for multi-label text classification. It attains Micro-F1 scores that either match or surpass those obtained with other frequently employed loss functions. It demonstrates a significant improvement in Macro-F1 scores across three multi-label datasets.
  arXiv  Detail & Related papers  (2024-04-12T11:12:16Z)
• From Pretext to Purpose: Batch-Adaptive Self-Supervised Learning [32.18543787821028]
  This paper proposes an adaptive technique of batch fusion for self-supervised contrastive learning. It achieves state-of-the-art performance under equitable comparisons. We suggest that the proposed method may contribute to the advancement of data-driven self-supervised learning research.
  arXiv  Detail & Related papers  (2023-11-16T15:47:49Z)
• Unlearn What You Want to Forget: Efficient Unlearning for LLMs [92.51670143929056]
  Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data. This process might suffer from privacy issues and violations of data protection regulations. We propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals.
  arXiv  Detail & Related papers  (2023-10-31T03:35:59Z)
• Efficient Vertical Federated Learning with Secure Aggregation [10.295508659999783]
  We present a novel design for training vertical FL securely and efficiently using state-of-the-art security modules for secure aggregation. We demonstrate empirically that our method does not impact training performance whilst obtaining 9.1e2 3.8e4 speedup compared to homomorphic encryption (HE)
  arXiv  Detail & Related papers  (2023-05-18T18:08:36Z)
• Responsible Active Learning via Human-in-the-loop Peer Study [88.01358655203441]
  We propose a responsible active learning method, namely Peer Study Learning (PSL), to simultaneously preserve data privacy and improve model stability. We first introduce a human-in-the-loop teacher-student architecture to isolate unlabelled data from the task learner (teacher) on the cloud-side. During training, the task learner instructs the light-weight active learner which then provides feedback on the active sampling criterion.
  arXiv  Detail & Related papers  (2022-11-24T13:18:27Z)
• Mitigating Forgetting in Online Continual Learning via Contrasting Semantically Distinct Augmentations [22.289830907729705]
  Online continual learning (OCL) aims to enable model learning from a non-stationary data stream to continuously acquire new knowledge as well as retain the learnt one. Main challenge comes from the "catastrophic forgetting" issue -- the inability to well remember the learnt knowledge while learning the new ones.
  arXiv  Detail & Related papers  (2022-11-10T05:29:43Z)
• SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning [168.89470249446023]
  We present SURF, a semi-supervised reward learning framework that utilizes a large amount of unlabeled samples with data augmentation. In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor. Our experiments demonstrate that our approach significantly improves the feedback-efficiency of the preference-based method on a variety of locomotion and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2022-03-18T16:50:38Z)
• Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases [57.90226879210227]
  FedCy is a semi-supervised learning (FSSL) method that combines FL and self-supervised learning to exploit a decentralized dataset of both labeled and unlabeled videos. We demonstrate significant performance gains over state-of-the-art FSSL methods on the task of automatic recognition of surgical phases.
  arXiv  Detail & Related papers  (2022-03-14T17:44:53Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)
• Ask-n-Learn: Active Learning via Reliable Gradient Representations for Image Classification [29.43017692274488]
  Deep predictive models rely on human supervision in the form of labeled training data. We propose Ask-n-Learn, an active learning approach based on gradient embeddings obtained using the pesudo-labels estimated in each of the algorithm.
  arXiv  Detail & Related papers  (2020-09-30T05:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.