Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack

• URL: http://arxiv.org/abs/2409.00960v2
• Date: Wed, 4 Sep 2024 10:58:26 GMT
• Title: Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack
• Authors: Guanzhong Chen, Zhenghan Qin, Mingxin Yang, Yajie Zhou, Tao Fan, Tianyu Du, Zenglin Xu,
• Abstract summary: BiSR is the first data reconstruction attack designed to target both the forward and backward propagation processes of split learning (SL) We propose BiSR, the first data reconstruction attack (DRA) designed to target both the forward and backward propagation processes of SL.
• Score: 20.727726850786386
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in pre-trained large language models (LLMs) have significantly influenced various domains. Adapting these models for specific tasks often involves fine-tuning (FT) with private, domain-specific data. However, privacy concerns keep this data undisclosed, and the computational demands for deploying LLMs pose challenges for resource-limited data holders. This has sparked interest in split learning (SL), a Model-as-a-Service (MaaS) paradigm that divides LLMs into smaller segments for distributed training and deployment, transmitting only intermediate activations instead of raw data. SL has garnered substantial interest in both industry and academia as it aims to balance user data privacy, model ownership, and resource challenges in the private fine-tuning of LLMs. Despite its privacy claims, this paper reveals significant vulnerabilities arising from the combination of SL and LLM-FT: the Not-too-far property of fine-tuning and the auto-regressive nature of LLMs. Exploiting these vulnerabilities, we propose Bidirectional Semi-white-box Reconstruction (BiSR), the first data reconstruction attack (DRA) designed to target both the forward and backward propagation processes of SL. BiSR utilizes pre-trained weights as prior knowledge, combining a learning-based attack with a bidirectional optimization-based approach for highly effective data reconstruction. Additionally, it incorporates a Noise-adaptive Mixture of Experts (NaMoE) model to enhance reconstruction performance under perturbation. We conducted systematic experiments on various mainstream LLMs and different setups, empirically demonstrating BiSR's state-of-the-art performance. Furthermore, we thoroughly examined three representative defense mechanisms, showcasing our method's capability to reconstruct private data even in the presence of these defenses.

Related papers

• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression. LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model. Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• Clear Minds Think Alike: What Makes LLM Fine-tuning Robust? A Study of Token Perplexity [61.48338027901318]
  We show that fine-tuning with LLM-generated data improves target task performance and reduces out-of-domain degradation. This is the first mechanistic explanation for the superior OOD robustness conferred by LLM-generated training data.
  arXiv  Detail & Related papers  (2025-01-24T08:18:56Z)
• Model Inversion in Split Learning for Personalized LLMs: New Insights from Information Bottleneck Theory [11.83473842859642]
  This work is the first to identify model inversion attacks in the split learning framework for personalized LLMs. We propose a two-stage attack system in which the first part projects representations into the embedding space, and the second part uses a generative model to recover text from these embeddings.
  arXiv  Detail & Related papers  (2025-01-10T13:47:13Z)
• Federated Fine-Tuning of LLMs: Framework Comparison and Research Directions [59.5243730853157]
  Federated learning (FL) provides a privacy-preserving solution for fine-tuning pre-trained large language models (LLMs) using distributed private datasets. This article conducts a comparative analysis of three advanced federated LLM (FedLLM) frameworks that integrate knowledge distillation (KD) and split learning (SL) to mitigate these issues.
  arXiv  Detail & Related papers  (2025-01-08T11:37:06Z)
• A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs [74.35290684163718]
  A primary challenge in large language model (LLM) development is their onerous pre-training cost. This paper explores a promising paradigm to improve LLM pre-training efficiency and quality by leveraging a small language model (SLM)
  arXiv  Detail & Related papers  (2024-10-24T14:31:52Z)
• Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation [50.837277466987345]
  We focus on the field of large language models (LLMs) for recommendation. We propose RecLoRA, which incorporates a Personalized LoRA module that maintains independent LoRAs for different users. We also design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces.
  arXiv  Detail & Related papers  (2024-08-07T04:20:28Z)
• PISTOL: Dataset Compilation Pipeline for Structural Unlearning of LLMs [31.16117964915814]
  Machine unlearning, which seeks to erase specific data stored in the pre-trained or fine-tuned models, has emerged as a crucial protective measure for LLMs. To facilitate the development of structural unlearning methods, we propose PISTOL, a pipeline for compiling multi-scenario datasets. We conduct benchmarks with four distinct unlearning methods on both Llama2-7B and Mistral-7B models.
  arXiv  Detail & Related papers  (2024-06-24T17:22:36Z)
• Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning [32.52811740662061]
  This article introduces DP-LoRA, a novel federated learning algorithm tailored for large language models (LLMs) DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training.
  arXiv  Detail & Related papers  (2023-12-29T06:50:38Z)
• Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning [50.9692060692705]
  This paper introduces $textbfLanguage Models for $textbfMo$tion Control ($textbfLaMo$), a general framework based on Decision Transformers for offline RL. Our framework highlights four crucial components:. Initializing Decision Transformers with sequentially pre-trained LMs, (2) employing the LoRA fine-tuning method,. In particular, our method demonstrates superior performance in scenarios with limited data samples.
  arXiv  Detail & Related papers  (2023-10-31T16:24:17Z)
• CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model [22.870512676002463]
  This paper focuses on Offsite-Tuning (OFT), a representative technique that transfers transformer blocks between centralized LLMs and downstream emulators. Inspired by these observations, we propose CRaSh, involving Clustering, Removing, and Sharing, a training-free strategy to derive improved emulators from LLMs. Our findings demonstrate a linear connectivity among these optima falling over the same basin, thereby highlighting the effectiveness of CRaSh and OFT.
  arXiv  Detail & Related papers  (2023-10-24T03:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.