TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction

• URL: http://arxiv.org/abs/2607.04484v1
• Date: Sun, 05 Jul 2026 20:11:00 GMT
• Title: TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction
• Abstract summary: TrustCLIP is a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary.<n>We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines.
• Score: 51.17509341761903
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision and vision-language models rely on high-level visual representations that are increasingly used across recognition, retrieval, and multimodal reasoning pipelines. However, recent advances in generative modeling have shown that such features can often be inverted, enabling realistic reconstructions of the underlying image and raising significant privacy risks. We revisit this problem through the lens of reconstruction and propose TrustCLIP, a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary. TrustCLIP learns a projection between encoder features and downstream modules that is explicitly optimized to degrade the reconstructions produced by generative attackers while retaining the necessary signals for downstream tasks. Unlike prior defenses that rely on discriminative privacy metrics, TrustCLIP directly optimizes against a generative reconstruction attacker, targeting a threat not captured by standard evaluation protocols. We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines. Across these settings, TrustCLIP consistently reduces the fidelity of generative inversions while maintaining downstream task performance. Project page: https://atnikos.github.io/trustclip/

Related papers

• What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference [13.194874392306085]
  We introduce ActInv, which solves an intermediate activation matching problem to reconstruct the client's input.<n>To understand this vulnerability, we develop Perturbation Amplification Factor (PAF), a metric for quantifying a layer's inherent resistance to reconstruction.<n>Our analysis reveals that privacy vulnerability is not uniform across layers, with some layers being highly susceptible to leakage while others offer natural resistance.
  arXiv  Detail & Related papers  (2026-05-22T02:14:16Z)
• ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recovery [73.8181449261685]
  Federated Learning (FL) enables collaborative model training by sharing model updates instead of raw data, aiming to protect user privacy.<n>Recent studies reveal that these shared updates can inadvertently leak sensitive training data through gradient inversion attacks (GIAs)
  arXiv  Detail & Related papers  (2026-03-18T11:40:44Z)
• TIP: Resisting Gradient Inversion via Targeted Interpretable Perturbation in Federated Learning [8.156452885913108]
  Federated Learning (FL) facilitates collaborative model training while preserving data locality.<n>The exchange of gradients renders the system vulnerable to Gradient Inversion Attacks (GIAs)<n>We propose Targeted Interpretable Perturbation (TIP), a novel defense framework that integrates model interpretability with frequency domain analysis.
  arXiv  Detail & Related papers  (2026-02-12T06:32:49Z)
• Deep Leakage with Generative Flow Matching Denoiser [54.05993847488204]
  We introduce a new deep leakage (DL) attack that integrates a generative Flow Matching (FM) prior into the reconstruction process.<n>Our approach consistently outperforms state-of-the-art attacks across pixel-level, perceptual, and feature-based similarity metrics.
  arXiv  Detail & Related papers  (2026-01-21T14:51:01Z)
• SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Models [67.84174763413178]
  We introduce SafeRedir, a lightweight inference-time framework for robust unlearning via prompt embedding redirection.<n>We show that SafeRedir achieves effective unlearning capability, high semantic and perceptual preservation, robust image quality, and enhanced resistance to adversarial attacks.
  arXiv  Detail & Related papers  (2026-01-13T15:01:38Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• Revisiting the Privacy Risks of Split Inference: A GAN-Based Data Reconstruction Attack via Progressive Feature Optimization [49.32786615205064]
  Split Inference (SI) partitions computation between edge devices and the cloud to reduce latency and protect user privacy.<n>Recent advances in Data Reconstruction Attacks (DRAs) reveal that intermediate features exchanged in SI can be exploited to recover sensitive input data.<n>Existing DRAs are typically effective only on shallow models and fail to fully leverage semantic priors.<n>We propose a novel GAN-based DRA framework with Progressive Feature Optimization (PFO), which decomposes the generator into hierarchical blocks and incrementally refines intermediate representations to enhance the semantic fidelity of reconstructed images.
  arXiv  Detail & Related papers  (2025-08-28T10:00:39Z)
• LeakyCLIP: Extracting Training Data from CLIP [23.702455444224995]
  textbfLeakyCLIP aims to achieve high-quality, semantically accurate image reconstruction from CLIP embeddings.<n>We identify three key challenges in CLIP inversion: 1) non-robust features, 2) limited visual semantics in text embeddings, and 3) low reconstruction fidelity.
  arXiv  Detail & Related papers  (2025-08-01T16:32:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.