Stealing Training Data from Large Language Models in Decentralized Training through Activation Inversion Attack

• URL: http://arxiv.org/abs/2502.16086v1
• Date: Sat, 22 Feb 2025 05:19:20 GMT
• Title: Stealing Training Data from Large Language Models in Decentralized Training through Activation Inversion Attack
• Authors: Chenxi Dai, Lin Lu, Pan Zhou,
• Abstract summary: Decentralized training has become a resource-efficient framework to democratize the training of large language models (LLMs)<n>This paper identifies a novel and realistic attack surface: the privacy leakage from training data in decentralized training.
• Score: 53.823990570014494
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decentralized training has become a resource-efficient framework to democratize the training of large language models (LLMs). However, the privacy risks associated with this framework, particularly due to the potential inclusion of sensitive data in training datasets, remain unexplored. This paper identifies a novel and realistic attack surface: the privacy leakage from training data in decentralized training, and proposes \textit{activation inversion attack} (AIA) for the first time. AIA first constructs a shadow dataset comprising text labels and corresponding activations using public datasets. Leveraging this dataset, an attack model can be trained to reconstruct the training data from activations in victim decentralized training. We conduct extensive experiments on various LLMs and publicly available datasets to demonstrate the susceptibility of decentralized training to AIA. These findings highlight the urgent need to enhance security measures in decentralized training to mitigate privacy risks in training LLMs.

Related papers

• FEDLAD: Federated Evaluation of Deep Leakage Attacks and Defenses [50.921333548391345]
  Federated Learning is a privacy preserving decentralized machine learning paradigm.<n>Recent research has revealed that private ground truth data can be recovered through a gradient technique known as Deep Leakage.<n>This paper introduces the FEDLAD Framework (Federated Evaluation of Deep Leakage Attacks and Defenses), a comprehensive benchmark for evaluating Deep Leakage attacks and defenses.
  arXiv  Detail & Related papers  (2024-11-05T11:42:26Z)
• Lancelot: Towards Efficient and Privacy-Preserving Byzantine-Robust Federated Learning within Fully Homomorphic Encryption [10.685816010576918]
  We propose Lancelot, an innovative and computationally efficient BRFL framework that employs fully homomorphic encryption (FHE) to safeguard against malicious client activities while preserving data privacy. Our extensive testing, which includes medical imaging diagnostics and widely-used public image datasets, demonstrates that Lancelot significantly outperforms existing methods, offering more than a twenty-fold increase in processing speed, all while maintaining data privacy.
  arXiv  Detail & Related papers  (2024-08-12T14:48:25Z)
• Defending against Data Poisoning Attacks in Federated Learning via User Elimination [0.0]
  This paper introduces a novel framework focused on the strategic elimination of adversarial users within a federated model. We detect anomalies in the aggregation phase of the Federated Algorithm, by integrating metadata gathered by the local training instances with Differential Privacy techniques. Our experiments demonstrate the efficacy of our methods, significantly mitigating the risk of data poisoning while maintaining user privacy and model performance.
  arXiv  Detail & Related papers  (2024-04-19T10:36:00Z)
• Leak and Learn: An Attacker's Cookbook to Train Using Leaked Data from Federated Learning [4.533760678036969]
  Federated learning is a decentralized learning paradigm introduced to preserve privacy of client data. Prior work has shown that an attacker can still reconstruct the private training data using only the client updates. We explore data reconstruction attacks through the lens of training and improve models with leaked data.
  arXiv  Detail & Related papers  (2024-03-26T23:05:24Z)
• Membership Information Leakage in Federated Contrastive Learning [7.822625013699216]
  Federated Contrastive Learning (FCL) represents a burgeoning approach for learning from decentralized unlabeled data. FCL is susceptible to privacy risks, such as membership information leakage, stemming from its distributed nature. This study delves into the feasibility of executing a membership inference attack on FCL and proposes a robust attack methodology.
  arXiv  Detail & Related papers  (2024-03-06T19:53:25Z)
• Amplifying Training Data Exposure through Fine-Tuning with Pseudo-Labeled Memberships [3.544065185401289]
  Neural language models (LMs) are vulnerable to training data extraction attacks due to data memorization. This paper introduces a novel attack scenario wherein an attacker fine-tunes pre-trained LMs to amplify the exposure of the original training data. Our empirical findings indicate a remarkable outcome: LMs with over 1B parameters exhibit a four to eight-fold increase in training data exposure.
  arXiv  Detail & Related papers  (2024-02-19T14:52:50Z)
• Exploring the Robustness of Decentralized Training for Large Language Models [51.41850749014054]
  Decentralized training of large language models has emerged as an effective way to democratize this technology. This paper explores the robustness of decentralized training from three main perspectives.
  arXiv  Detail & Related papers  (2023-12-01T04:04:03Z)
• Transferable Unlearnable Examples [63.64357484690254]
  Un unlearnable strategies have been introduced to prevent third parties from training on the data without permission. They add perturbations to the users' data before publishing, which aims to make the models trained on the published dataset invalidated. We propose a novel unlearnable strategy based on Classwise Separability Discriminant (CSD), which aims to better transfer the unlearnable effects to other training settings and datasets.
  arXiv  Detail & Related papers  (2022-10-18T19:23:52Z)
• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses [150.64470864162556]
  This work systematically categorizes and discusses a wide range of dataset vulnerabilities and exploits. In addition to describing various poisoning and backdoor threat models and the relationships among them, we develop their unified taxonomy.
  arXiv  Detail & Related papers  (2020-12-18T22:38:47Z)
• Toward Smart Security Enhancement of Federated Learning Networks [109.20054130698797]
  In this paper, we review the vulnerabilities of federated learning networks (FLNs) and give an overview of poisoning attacks. We present a smart security enhancement framework for FLNs. Deep reinforcement learning is applied to learn the behaving patterns of the edge devices (EDs) that can provide benign training results.
  arXiv  Detail & Related papers  (2020-08-19T08:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.