What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference

• URL: http://arxiv.org/abs/2605.23158v1
• Date: Fri, 22 May 2026 02:14:16 GMT
• Title: What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference
• Abstract summary: We introduce ActInv, which solves an intermediate activation matching problem to reconstruct the client's input.<n>To understand this vulnerability, we develop Perturbation Amplification Factor (PAF), a metric for quantifying a layer's inherent resistance to reconstruction.<n>Our analysis reveals that privacy vulnerability is not uniform across layers, with some layers being highly susceptible to leakage while others offer natural resistance.
• Score: 13.194874392306085
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The deployment of large language models (LLMs) on resource-constrained devices remains challenging, spurring interest in split inference, where models are partitioned between client and server to reduce computational burden and enhance privacy by transmitting only intermediate activations. However, the privacy-preserving capabilities of split inference, particularly in the context of LLMs, have not been exhaustively investigated. To fill this gap, we introduce ActInv, which solves an intermediate activation matching problem to reconstruct the client's input. Extensive evaluations demonstrate that ActInv achieves high-fidelity reconstructions, even in the presence of common perturbation-based defenses such as Gaussian noise injection and activation sparsification. To systematically understand this vulnerability, we develop Perturbation Amplification Factor (PAF), a metric for quantifying a layer's inherent resistance to reconstruction. Our analysis reveals that privacy vulnerability is not uniform across layers, with some layers being highly susceptible to leakage while others offer natural resistance. Furthermore, we demonstrate that defense effectiveness can be significantly improved by calibrating perturbation directions to maximize reconstruction error during backpropagation. Building on these insights, we design PriPert and conduct comprehensive evaluations, covering privacy, utility, and computational overhead, to demonstrate its effectiveness.

Related papers

• TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction [51.17509341761903]
  TrustCLIP is a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary.<n>We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines.
  arXiv  Detail & Related papers  (2026-07-05T20:11:00Z)
• Defense Against Prompt Inversion Attacks: An Information-Theoretic Approach for LLM Collaborative Inference [5.218013327284395]
  Collaborative edge-cloud inference enables resource-constrained devices to leverage large language models (LLMs) by offloading partial computation to cloud servers.<n> transmitting intermediate activations exposes sensitive user prompts to prompt inversion attacks.<n>We propose an information-theoretic defense framework for prompt inversion in collaborative LLM inference.
  arXiv  Detail & Related papers  (2026-06-10T02:36:26Z)
• Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization [51.33055188976282]
  adversarial attacks have emerged as a prominent defense strategy to prevent malicious content forgery in personalized image generation.<n>We introduce Two-Stage Latent Feature Optimization (TS-LFO), an efficient and effective copyright-stealing attack against protected diffusion-based customization.<n>Extensive experiments show that TS-LFO consistently bypasses state-of-the-art (SOTA) copyright defenses and outperforms SOTA copyright attacks such as DiffPure, GrIDPure and IMPRESS.
  arXiv  Detail & Related papers  (2026-06-06T07:59:08Z)
• CausShield: Sample Reconstruction-Resilient Vertical FL via Causal Representation Learning [11.295359490102522]
  VFL is a distributed learning paradigm that leverages vertically partitioned features across isolated parties without sharing raw samples.<n>Existing defenses fail to achieve a satisfactory trade-off between model utility and privacy protection.<n>We introduce CausShield, which decomposes the shared representations between the client and the coordinating server in VFL into task-relevant and task-irrelevant components.
  arXiv  Detail & Related papers  (2026-06-06T07:40:11Z)
• ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recovery [73.8181449261685]
  Federated Learning (FL) enables collaborative model training by sharing model updates instead of raw data, aiming to protect user privacy.<n>Recent studies reveal that these shared updates can inadvertently leak sensitive training data through gradient inversion attacks (GIAs)
  arXiv  Detail & Related papers  (2026-03-18T11:40:44Z)
• NeuroFilter: Privacy Guardrails for Conversational LLM Agents [50.75206727081996]
  This work addresses the computational challenge of enforcing privacy for agentic Large Language Models (LLMs)<n>NeuroFilter is a guardrail framework that operationalizes contextual integrity by mapping norm violations to simple directions in the model's activation space.<n>A comprehensive evaluation across over 150,000 interactions, covering models from 7B to 70B parameters, illustrates the strong performance of NeuroFilter.
  arXiv  Detail & Related papers  (2026-01-21T05:16:50Z)
• DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage Mitigation in Text-to-Image Models [55.30555646945055]
  Text-to-Image (T2I) models are vulnerable to semantic leakage.<n>We introduce DeLeaker, a lightweight approach that mitigates leakage by directly intervening on the model's attention maps.<n>SLIM is the first dataset dedicated to semantic leakage.
  arXiv  Detail & Related papers  (2025-10-16T17:39:21Z)
• Revisiting the Privacy Risks of Split Inference: A GAN-Based Data Reconstruction Attack via Progressive Feature Optimization [49.32786615205064]
  Split Inference (SI) partitions computation between edge devices and the cloud to reduce latency and protect user privacy.<n>Recent advances in Data Reconstruction Attacks (DRAs) reveal that intermediate features exchanged in SI can be exploited to recover sensitive input data.<n>Existing DRAs are typically effective only on shallow models and fail to fully leverage semantic priors.<n>We propose a novel GAN-based DRA framework with Progressive Feature Optimization (PFO), which decomposes the generator into hierarchical blocks and incrementally refines intermediate representations to enhance the semantic fidelity of reconstructed images.
  arXiv  Detail & Related papers  (2025-08-28T10:00:39Z)
• Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems [89.35169042718739]
  collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers.<n>Recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs)<n>This work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA.<n>Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy algorithm to enhance the inversion robustness
  arXiv  Detail & Related papers  (2025-03-01T07:15:21Z)
• How Breakable Is Privacy: Probing and Resisting Model Inversion Attacks in Collaborative Inference [13.453033795109155]
  Collaborative inference improves computational efficiency for edge devices by transmitting intermediate features to cloud models.<n>There is no established criterion for assessing the difficulty of model inversion attacks (MIAs)<n>We propose the first theoretical criterion to assess MIA difficulty in CI, identifying mutual information, entropy, and effective information volume as key influencing factors.
  arXiv  Detail & Related papers  (2025-01-01T13:00:01Z)
• PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks [15.257413246220032]
  Collaborative inference is a promising solution to enable resource-constrained edge devices to perform inference using state-of-the-art deep neural networks (DNNs) Recent research indicates model inversion attacks (MIAs) can reconstruct input data from intermediate results, posing serious privacy concerns for collaborative inference. This paper provides a viable solution, named PATROL, which develops privacy-oriented pruning to balance privacy, efficiency, and utility of collaborative inference.
  arXiv  Detail & Related papers  (2023-07-20T16:09:07Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.