Inverting the Hidden: Unveiling Multimodal Privacy Leakage in Collaborative LVLM Inference

• URL: http://arxiv.org/abs/2608.01020v1
• Date: Sun, 02 Aug 2026 05:56:23 GMT
• Title: Inverting the Hidden: Unveiling Multimodal Privacy Leakage in Collaborative LVLM Inference
• Abstract summary: Collaborative inference deploys Large Vision-Language Models (LVLMs) by computation between edge devices and the cloud.<n>While withholding raw inputs supposedly ensures privacy, transmitting intermediate hidden states exposes a critical attack surface.<n>We show that privacy-sensitive visual and textual information can be recovered even from deep-layer LVLM hidden states.
• Score: 10.251138435714147
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative inference deploys Large Vision-Language Models (LVLMs) by partitioning computation between edge devices and the cloud. While withholding raw inputs supposedly ensures privacy, transmitting intermediate hidden states exposes a critical attack surface. However, it remains unclear whether deep-layer LVLM hidden states retain recoverable private information, given that visual content has been projected into the language embedding space. To address this concern, we theoretically analyze LVLM hidden-state recoverability and show that, under regularity assumptions and a positive semantic--nuisance margin, privacy-relevant visual semantics remain identifiable and stably recoverable. Motivated by this analysis, we propose RASR, a novel coarse-to-fine multimodal reconstruction attack. RASR obtains initial image and text reconstructions through modality-specific inverse paths that follow their respective forward processing pipelines in reverse, and then uses hidden-state consistency to refine both reconstructions. Evaluations on Qwen3-VL-8B-Instruct and LLaVA-1.5-7B across five datasets demonstrate that RASR reduces image reconstruction MSE by \(\sim\)50\% compared to the strongest baselines, while achieving up to 99\% token accuracy for text recovery. These results show that privacy-sensitive visual and textual information can be recovered even from deep-layer LVLM hidden states, exposing the privacy risks of collaborative inference.

Related papers

• Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection [41.47359176400441]
  Vision Language Models (VLMs) offer powerful multimodal ability but expose users to text-based privacy attacks.<n>We propose CloakDiff, the first framework for reversible, high fidelity privacy protection against text-based query attacks.
  arXiv  Detail & Related papers  (2026-07-11T14:13:10Z)
• TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction [51.17509341761903]
  TrustCLIP is a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary.<n>We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines.
  arXiv  Detail & Related papers  (2026-07-05T20:11:00Z)
• What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference [13.194874392306085]
  We introduce ActInv, which solves an intermediate activation matching problem to reconstruct the client's input.<n>To understand this vulnerability, we develop Perturbation Amplification Factor (PAF), a metric for quantifying a layer's inherent resistance to reconstruction.<n>Our analysis reveals that privacy vulnerability is not uniform across layers, with some layers being highly susceptible to leakage while others offer natural resistance.
  arXiv  Detail & Related papers  (2026-05-22T02:14:16Z)
• ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models [59.94664910790462]
  ResPrune is a training-free visual token pruning framework for large vision-language models.<n>It selects a compact yet informative subset of visual tokens.<n>It achieves effective reductions in computation, memory consumption, and inference latency.
  arXiv  Detail & Related papers  (2026-03-22T07:44:45Z)
• Revis: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models [18.624825577944854]
  REVIS is a training-free framework designed to explicitly re-activate suppressed visual information.<n>We show that REVIS reduces object hallucination rates by approximately 19% compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-12T11:07:44Z)
• NeuroFilter: Privacy Guardrails for Conversational LLM Agents [50.75206727081996]
  This work addresses the computational challenge of enforcing privacy for agentic Large Language Models (LLMs)<n>NeuroFilter is a guardrail framework that operationalizes contextual integrity by mapping norm violations to simple directions in the model's activation space.<n>A comprehensive evaluation across over 150,000 interactions, covering models from 7B to 70B parameters, illustrates the strong performance of NeuroFilter.
  arXiv  Detail & Related papers  (2026-01-21T05:16:50Z)
• The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitization [53.51921540246166]
  We show that Language Large Models (LLMs) can exploit the contextual vulnerability of DP-sanitized texts.<n>Experiments uncover a double-edged sword effect of LLM reconstructions on privacy and utility.<n>We propose recommendations for using data reconstruction as a post-processing step.
  arXiv  Detail & Related papers  (2025-08-26T12:22:45Z)
• CapRecover: A Cross-Modality Feature Inversion Attack Framework on Vision Language Models [11.201984255831126]
  CapRecover recovers high-level semantic content, such as labels or captions, directly from intermediate features without image reconstruction.<n>We introduce a simple yet effective protection method: adding random noise to intermediate features at each layer and removing the noise in the next layer.
  arXiv  Detail & Related papers  (2025-07-30T16:42:02Z)
• Safeguarding LLM Embeddings in End-Cloud Collaboration via Entropy-Driven Perturbation [16.419373701694067]
  EntroGuard is an entropy-driven embedding privacy protection method.<n>It can protect the privacy of text embeddings while maintaining retrieval accuracy during the end-cloud collaboration.
  arXiv  Detail & Related papers  (2025-03-17T07:58:05Z)
• TrojVLM: Backdoor Attack Against Vision Language Models [50.87239635292717]
  This study introduces TrojVLM, the first exploration of backdoor attacks aimed at Vision Language Models (VLMs) TrojVLM inserts predetermined target text into output text when encountering poisoned images. A novel semantic preserving loss is proposed to ensure the semantic integrity of the original image content.
  arXiv  Detail & Related papers  (2024-09-28T04:37:09Z)
• Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models [58.17315970207874]
  We propose a zero-shot method for adapting generalisable visual-textual priors from arbitrary VLM to facilitate moment-text alignment. Experiments conducted on three VMR benchmark datasets demonstrate the notable performance advantages of our zero-shot algorithm.
  arXiv  Detail & Related papers  (2023-09-01T13:06:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.