TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization

• URL: http://arxiv.org/abs/2606.18312v1
• Date: Tue, 16 Jun 2026 10:24:40 GMT
• Title: TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization
• Abstract summary: Federated learning allows multiple clients to jointly train a shared model by sending updates to a central server while keeping raw inputs local.<n>Prior gradient inversion attacks show that these updates can reveal enough information to reconstruct client inputs.<n>We introduce TIGER, a continuous gradient inversion attack that turns this subspace signal into a differentiable objective.
• Score: 4.486500523404418
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning allows multiple clients to jointly train a shared model by sending gradient updates to a central server while keeping raw inputs local. However, prior gradient inversion attacks show that these updates can reveal enough information to reconstruct client inputs. Existing attacks on transformers either optimize dummy inputs to match the true client updates, which is costly and unstable for modern models, or exploit the low rank of attention gradients to identify a subspace containing the true layer embeddings, followed by a discrete membership test for candidate tokens. However, this token test is brittle under numerical noise, i.e., from quantization or Differential Privacy (DP), and scales poorly for encoder models with non-causal attention. We introduce TIGER, a continuous gradient inversion attack that turns this subspace signal into a differentiable objective. Instead of searching over tokens or matching full gradients, TIGER directly optimizes token embeddings to minimize their distance to the subspace. Our experiments demonstrate that on encoder-only models, TIGER substantially improves both reconstruction quality and runtime over existing attacks, while on decoder models, TIGER is more robust than prior subspace-based attacks, enabling the first successful reconstructions in DP-defended federated learning settings.

Related papers

• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• Mitigating Gradient Inversion Risks in Language Models via Token Obfuscation [21.00172111387017]
  Training and fine-tuning large-scale language models are vulnerable to gradient inversion attacks (GIAs)<n>GHOST is a token-level obfuscation mechanism that neutralizes GIAs by decoupling the inherent connections across gradient, embedding, and token spaces.<n>GHOST is built upon an important insight: due to the large scale of the token space, there exist semantically distinct yet embedding-proximate tokens that can serve as the shadow substitutes of the original tokens.
  arXiv  Detail & Related papers  (2026-02-11T14:49:31Z)
• CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling [63.07948989346385]
  Federated learning collaboratively trains a neural network on a global server.<n>Each local client receives the current global model weights and sends back parameter updates (gradients) based on its local private data.<n>Existing gradient inversion attacks can exploit this vulnerability to recover private training instances from a client's gradient vectors.<n>We present a novel defense tailored for large neural network models.
  arXiv  Detail & Related papers  (2025-01-27T01:06:23Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [81.93945602120453]
  We introduce an approach that is both general and parameter-efficient for face forgery detection.<n>We design a forgery-style mixture formulation that augments the diversity of forgery source domains.<n>We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• DAGER: Exact Gradient Inversion for Large Language Models [10.998375857698496]
  Federated learning works by aggregating locally computed gradients from multiple clients. Prior work has shown that the data can actually be recovered by the server using so-called gradient inversion attacks. We propose DAGER, the first algorithm to recover whole batches of input text exactly.
  arXiv  Detail & Related papers  (2024-05-24T14:14:24Z)
• Rethinking PGD Attack: Is Sign Function Necessary? [131.6894310945647]
  We present a theoretical analysis of how such sign-based update algorithm influences step-wise attack performance. We propose a new raw gradient descent (RGD) algorithm that eliminates the use of sign. The effectiveness of the proposed RGD algorithm has been demonstrated extensively in experiments.
  arXiv  Detail & Related papers  (2023-12-03T02:26:58Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• Subspace based Federated Unlearning [75.90552823500633]
  Federated unlearning (FL) aims to remove a specified target client's contribution in FL to satisfy the user's right to be forgotten. Most existing federated unlearning algorithms require the server to store the history of the parameter updates. We propose a simple-yet-effective subspace based federated unlearning method, dubbed SFU, that lets the global model perform gradient ascent.
  arXiv  Detail & Related papers  (2023-02-24T04:29:44Z)
• Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks [19.917677500613788]
  gradient-based approaches generally use the $sign$ function to generate perturbations at the end of the process. We propose a Sampling-based Fast Gradient Rescaling Method (S-FGRM) to improve the transferability of crafted adversarial examples.
  arXiv  Detail & Related papers  (2022-04-06T15:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.