FISGuard: Defending Against Membership Inference via Fixed Input Subspaces

• URL: http://arxiv.org/abs/2608.27836v1
• Date: Fri, 28 Aug 2026 02:06:37 GMT
• Title: FISGuard: Defending Against Membership Inference via Fixed Input Subspaces
• Abstract summary: FISGuard is a lightweight defense against membership inference attacks.<n>It fixes a low-dimensional representation subspace using independent public data.<n>It reduces the ProjRes attack AUC to near the random-guessing level of 0.5 in most settings.
• Score: 5.930798548700099
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As large language models are increasingly adopted in federated learning, protecting user privacy while performing parameter-efficient fine-tuning on distributed private data has become an important challenge. Although clients only share gradients instead of directly uploading raw data, the shared gradients may still leak membership information about training samples. ProjRes (S&P, 2026) further increases this risk: with less information and without accessing model outputs, an attacker can effectively distinguish members from non-members solely based on the projection residual between a candidate representation and the subspace induced by server-observable gradients. Existing defenses against membership inference mostly rely on gradient perturbation or regularization, which can not only degrade model utility but also fail to effectively defend against the membership inference attack introduced by ProjRes, which exploits the geometric structure of gradients. To address this issue, we propose FISGuard, a lightweight defense. Its key idea is to construct and fix a low-dimensional representation subspace using independent public data, thereby restricting the space through which private representations are exposed via gradients while preserving the primary information required for downstream tasks. This substantially reduces the projection-residual discrepancy between members and non-members. We evaluate FISGuard against five representative defense methods across three NLP datasets, two LLMs, and two fine-tuning strategies, Adapter and LoRA. The results show that FISGuard reduces the ProjRes attack AUC to near the random-guessing level of 0.5 in most settings, while maintaining downstream task performance close to that of the undefended model and introducing only limited computational overhead, thereby achieving a favorable privacy--utility trade-off.

Related papers

• LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections [54.51153206716027]
  Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server.<n>We propose LightSplit, which limits information exposure and reduces communication overhead by applying a lightweight fixed random projection at the cut layer.<n>Our results show that the method retains more than 95% of the baseline accuracy at up to 32x reduction in transmitted dimensionality.
  arXiv  Detail & Related papers  (2026-05-13T09:45:49Z)
• Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach [44.738736515079346]
  Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data.<n>FedLLMs' unique properties, i.e. massive parameter scales, rapid convergence, and sparse, non-orthogonal gradients, render existing MIAs ineffective.<n>We propose ProjRes, the first projection residuals-based passive MIA tailored for FedLLMs.
  arXiv  Detail & Related papers  (2026-04-23T01:44:04Z)
• G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMs [1.8986796884429726]
  Membership inference attacks (MIAs) aim to determine whether a given example was used during training.<n>We introduce G-Drift MIA, a white-box membership inference method based on gradient-induced feature drift.<n>G-Drift substantially outperforms confidence-based, perplexity-based, and reference-based attacks.
  arXiv  Detail & Related papers  (2026-04-01T03:01:27Z)
• Teleportation-Based Defenses for Privacy in Approximate Machine Unlearning [8.735490611482364]
  Approximate machine unlearning aims to efficiently remove the influence of specific data points from a trained model.<n>An adversary with access to pre- and post-unlearning models can exploit their differences for membership inference or data reconstruction.<n>We show these vulnerabilities arise from two factors: large gradient norms of forget-set samples and the close proximity of unlearned parameters to the original model.
  arXiv  Detail & Related papers  (2025-11-29T01:50:33Z)
• FL-PLAS: Federated Learning with Partial Layer Aggregation for Backdoor Defense Against High-Ratio Malicious Clients [7.1383449614815415]
  Federated learning (FL) is gaining increasing attention as an emerging collaborative machine learning approach.<n>The fundamental algorithm of FL, Federated Averaging (FedAvg), is susceptible to backdoor attacks.<n>We propose a novel defense algorithm, FL-PLAS, which can effectively protect local models from backdoor attacks.
  arXiv  Detail & Related papers  (2025-05-17T14:16:47Z)
• CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling [63.07948989346385]
  Federated learning collaboratively trains a neural network on a global server.<n>Each local client receives the current global model weights and sends back parameter updates (gradients) based on its local private data.<n>Existing gradient inversion attacks can exploit this vulnerability to recover private training instances from a client's gradient vectors.<n>We present a novel defense tailored for large neural network models.
  arXiv  Detail & Related papers  (2025-01-27T01:06:23Z)
• Understanding Deep Gradient Leakage via Inversion Influence Functions [53.1839233598743]
  Deep Gradient Leakage (DGL) is a highly effective attack that recovers private training images from gradient vectors. We propose a novel Inversion Influence Function (I$2$F) that establishes a closed-form connection between the recovered images and the private gradients. We empirically demonstrate that I$2$F effectively approximated the DGL generally on different model architectures, datasets, attack implementations, and perturbation-based defenses.
  arXiv  Detail & Related papers  (2023-09-22T17:26:24Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• Subspace based Federated Unlearning [75.90552823500633]
  Federated unlearning (FL) aims to remove a specified target client's contribution in FL to satisfy the user's right to be forgotten. Most existing federated unlearning algorithms require the server to store the history of the parameter updates. We propose a simple-yet-effective subspace based federated unlearning method, dubbed SFU, that lets the global model perform gradient ascent.
  arXiv  Detail & Related papers  (2023-02-24T04:29:44Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• Concealing Sensitive Samples against Gradient Leakage in Federated Learning [41.43099791763444]
  Federated Learning (FL) is a distributed learning paradigm that enhances users privacy by eliminating the need for clients to share raw, private data with the server. Recent studies expose the vulnerability of FL to model inversion attacks, where adversaries reconstruct users private data via eavesdropping on the shared gradient information. We present a simple, yet effective defense strategy that obfuscates the gradients of the sensitive data with concealed samples.
  arXiv  Detail & Related papers  (2022-09-13T04:19:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.