AEGIS: Attention-Embedding Gradient Isolation Shield - Triple-Channel Gradient Masking for Privacy-Preserving Federated LLM Fine-Tuning

• URL: http://arxiv.org/abs/2608.19534v1
• Date: Thu, 20 Aug 2026 01:12:56 GMT
• Title: AEGIS: Attention-Embedding Gradient Isolation Shield - Triple-Channel Gradient Masking for Privacy-Preserving Federated LLM Fine-Tuning
• Abstract summary: Gradient inversion attacks recover private training text from gradients shared in federated learning.<n>AEGIS is a lightweight defence that closes all three analytical channels with three backward-path operations.<n>It reduces token recovery rates to near zero against a range of gradient inversion attacks.
• Score: 13.743991513761273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gradient inversion attacks recover private training text from gradients shared in federated learning, posing a serious threat to collaborative model training. Through our analysis of transformer gradient structure, we identify three channels through which private token information leaks: the attention output projection gradient exposes a low-rank subspace that encodes input embeddings (Channel 1), the embedding gradient's row-norm sparsity directly reveals which tokens are present (Channel 2), and the MLP expansion gradient carries a recoverable subspace signal analogous to Channel 1 (Channel 3). State-of-the-art attacks exploit these channels analytically to achieve near-exact token recovery in seconds. Existing defences address at most one channel and either degrade model utility or leave the remaining structural signals intact. We introduce AEGIS (Attention-Embedding Gradient Isolation Shield), a lightweight defence that closes all three analytical channels with three backward-path operations requiring no architectural changes: freezing attention projection parameters eliminates Channel 1 by construction, calibrated noise injection into the embedding gradient destroys Channel 2's token-presence signal, and analogous per-block noise injection into the MLP expansion gradient masks Channel 3. The same masked gradient drives both the local optimiser step and the server export, so no clean signal is retained on either side. Evaluated across 11 models and six datasets, AEGIS reduces token recovery rates to near zero against a range of gradient inversion attacks, both analytical and optimisation-based, while preserving or improving model utility. We provide formal guarantees for Channels 1 and 2 and validate the full defence empirically against adaptive adversaries with complete knowledge of the mechanism.

Related papers

• SignDeepSC: A Semantic Signature-based Approach for Robust Semantic Communication [4.042562775811428]
  SignDeepSC is an architectural defense that achieves adversarial robustness without requiring adversarial example generation during training.<n>System is built on a perceiver-inspired semantic signature, a compact vector summary of the source features transmitted over a separate low-rate auxiliary channel.
  arXiv  Detail & Related papers  (2026-07-28T12:54:13Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models [22.960033047333024]
  Self-supervised speech models (S3Ms) achieve strong downstream performance, yet their learned representations remain poorly understood under natural and adversarial perturbations.<n>We present GRIDS, a framework using Local Intrinsic Dimensionality (LID) across layer-wise representations in WavLM and wav2vec 2.0.
  arXiv  Detail & Related papers  (2026-05-04T15:18:15Z)
• AEGIS: Anchor-Enforced Gradient Isolation for Knowledge-Preserving Vision-Language-Action Fine-Tuning [0.0]
  Adapting pre-trained vision-language models (VLMs) for robotic control requires injecting high-magnitude continuous gradients from a flow-matching action expert into a backbone trained exclusively with cross-entropy.<n>This cross-modal gradient asymmetry causes rapid, severe erosion of the VLM's visual-question-answering (VQA) capability.<n>Industry-standard defences either sever the gradient pathway entirely via stop gradient, discarding the rich continuous supervision, or restrict parameter capacity through low-rank adapters (LoRA) that constrain the rank of updates but not their direction, and thus still overwrite the pre-trained manifold
  arXiv  Detail & Related papers  (2026-04-17T13:49:57Z)
• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• Model Compression using Progressive Channel Pruning [73.95107098134964]
  We propose a framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs)<n>Our framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration.<n>Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches.
  arXiv  Detail & Related papers  (2025-07-07T09:12:03Z)
• Mjolnir: Breaking the Shield of Perturbation-Protected Gradients via Adaptive Diffusion [13.764770382623812]
  We present the first attempt to break the shield of gradient perturbation protection in Federated Learning.<n>We introduce Mjolnir, a perturbation-resilient gradient leakage attack.<n>Mjolnir is capable of removing perturbations from gradients without requiring additional access to the original model structure or external data.
  arXiv  Detail & Related papers  (2024-07-07T07:06:49Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• Locality-aware Channel-wise Dropout for Occluded Face Recognition [116.2355331029041]
  Face recognition is a challenging task in unconstrained scenarios, especially when faces are partially occluded. We propose a novel and elegant occlusion-simulation method via dropping the activations of a group of neurons in some elaborately selected channel. Experiments on various benchmarks show that the proposed method outperforms state-of-the-art methods with a remarkable improvement.
  arXiv  Detail & Related papers  (2021-07-20T05:53:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.