Model Inversion in Split Learning for Personalized LLMs: New Insights from Information Bottleneck Theory

• URL: http://arxiv.org/abs/2501.05965v1
• Date: Fri, 10 Jan 2025 13:47:13 GMT
• Title: Model Inversion in Split Learning for Personalized LLMs: New Insights from Information Bottleneck Theory
• Authors: Yunmeng Shu, Shaofeng Li, Tian Dong, Yan Meng, Haojin Zhu,
• Abstract summary: This work is the first to identify model inversion attacks in the split learning framework for personalized LLMs.<n>We propose a two-stage attack system in which the first part projects representations into the embedding space, and the second part uses a generative model to recover text from these embeddings.
• Score: 11.83473842859642
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized Large Language Models (LLMs) have become increasingly prevalent, showcasing the impressive capabilities of models like GPT-4. This trend has also catalyzed extensive research on deploying LLMs on mobile devices. Feasible approaches for such edge-cloud deployment include using split learning. However, previous research has largely overlooked the privacy leakage associated with intermediate representations transmitted from devices to servers. This work is the first to identify model inversion attacks in the split learning framework for LLMs, emphasizing the necessity of secure defense. For the first time, we introduce mutual information entropy to understand the information propagation of Transformer-based LLMs and assess privacy attack performance for LLM blocks. To address the issue of representations being sparser and containing less information than embeddings, we propose a two-stage attack system in which the first part projects representations into the embedding space, and the second part uses a generative model to recover text from these embeddings. This design breaks down the complexity and achieves attack scores of 38%-75% in various scenarios, with an over 60% improvement over the SOTA. This work comprehensively highlights the potential privacy risks during the deployment of personalized LLMs on the edge side.

Related papers

• PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing [48.30406812516552]
  We introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimize model architecture and edge system constraints. PLM employs a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint. evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data.
  arXiv  Detail & Related papers  (2025-03-15T15:11:17Z)
• Prompt Inference Attack on Distributed Large Language Model Inference Frameworks [22.787604216912257]
  Inference process of modern large language models (LLMs) demands prohibitive computational resources, rendering them infeasible for deployment on consumer-grade devices. Recent studies propose distributed LLM inference frameworks, which employ split learning principles to enable collaborative LLM inference on resource-constrained hardware. However, distributing LLM layers across participants requires the transmission of intermediate outputs, which may introduce privacy risks to the original input prompts.
  arXiv  Detail & Related papers  (2025-03-12T11:36:29Z)
• Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack [20.727726850786386]
  BiSR is the first data reconstruction attack designed to target both the forward and backward propagation processes of split learning (SL) We propose BiSR, the first data reconstruction attack (DRA) designed to target both the forward and backward propagation processes of SL.
  arXiv  Detail & Related papers  (2024-09-02T06:01:20Z)
• LLM-PBE: Assessing Data Privacy in Large Language Models [111.58198436835036]
  Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs.
  arXiv  Detail & Related papers  (2024-08-23T01:37:29Z)
• Evaluating LLM-based Personal Information Extraction and Countermeasures [63.91918057570824]
  Large language model (LLM) based personal information extraction can be benchmarked. LLM can be misused by attackers to accurately extract various personal information from personal profiles. prompt injection can defend against strong LLM-based attacks, reducing the attack to less effective traditional ones.
  arXiv  Detail & Related papers  (2024-08-14T04:49:30Z)
• Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs [61.04246774006429]
  We introduce a black-box prompt optimization method that uses an attacker LLM agent to uncover higher levels of memorization in a victim agent. We observe that our instruction-based prompts generate outputs with 23.7% higher overlap with training data compared to the baseline prefix-suffix measurements. Our findings show that instruction-tuned models can expose pre-training data as much as their base-models, if not more so, and using instructions proposed by other LLMs can open a new avenue of automated attacks.
  arXiv  Detail & Related papers  (2024-03-05T19:32:01Z)
• Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models [4.081098869497239]
  We develop state-of-the-art privacy attacks against Large Language Models (LLMs) New membership inference attacks (MIAs) against pretrained LLMs perform hundreds of times better than baseline attacks. In fine-tuning, we find that a simple attack based on the ratio of the loss between the base and fine-tuned models is able to achieve near-perfect MIA performance.
  arXiv  Detail & Related papers  (2024-02-26T20:41:50Z)
• Knowledge Fusion of Large Language Models [73.28202188100646]
  This paper introduces the notion of knowledge fusion for large language models (LLMs) We externalize their collective knowledge and unique strengths, thereby elevating the capabilities of the target model beyond those of any individual source LLM. Our findings confirm that the fusion of LLMs can improve the performance of the target model across a range of capabilities such as reasoning, commonsense, and code generation.
  arXiv  Detail & Related papers  (2024-01-19T05:02:46Z)
• Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities [32.035405009895264]
  Large language models (LLMs) are revolutionizing AI development and potentially shaping our future. The status quo cloud-based deployment faces some critical challenges: 1) long response time; 2) high bandwidth costs; and 3) the violation of data privacy. 6G mobile edge computing (MEC) systems may resolve these pressing issues. This article serves as a position paper for thoroughly identifying the motivation, challenges, and pathway for empowering LLMs at the 6G edge.
  arXiv  Detail & Related papers  (2023-09-28T06:22:59Z)
• Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey [43.063650238194384]
  Large Language Models (LLMs) have shown greatly enhanced performance in recent years, attributed to increased size and extensive training data. Training data memorization in Machine Learning models scales with model size, particularly concerning for LLMs. Memorized text sequences have the potential to be directly leaked from LLMs, posing a serious threat to data privacy.
  arXiv  Detail & Related papers  (2023-09-27T15:15:23Z)
• LLM-Pruner: On the Structural Pruning of Large Language Models [65.02607075556742]
  Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. We tackle the compression of LLMs within the bound of two constraints: being task-agnostic and minimizing the reliance on the original training dataset. Our method, named LLM-Pruner, adopts structural pruning that selectively removes non-critical coupled structures.
  arXiv  Detail & Related papers  (2023-05-19T12:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.