Taming Noise-Induced Prototype Degradation for Privacy-Preserving Personalized Federated Fine-Tuning

• URL: http://arxiv.org/abs/2604.27833v1
• Date: Thu, 30 Apr 2026 13:20:51 GMT
• Title: Taming Noise-Induced Prototype Degradation for Privacy-Preserving Personalized Federated Fine-Tuning
• Abstract summary: Prototype-based Personalized Federated Learning (ProtoPFL) enables efficient multi-domain adaptation by communicating compact class prototypes.<n>A common defense involves per-example $ell$ clipping before prototype to bound sensitivity, followed by isotropic noise to enforce Local Differential Privacy.<n>We propose VPDR, a client-side privacy plug-in that seamlessly integrates into existing ProtoPFL.
• Score: 21.950552296952623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prototype-based Personalized Federated Learning (ProtoPFL) enables efficient multi-domain adaptation by communicating compact class prototypes, but directly sharing them poses privacy risks. A common defense involves per-example $\ell_2$ clipping before prototype computation to bound sensitivity, followed by isotropic Gaussian noise to enforce Local Differential Privacy (LDP). However, Isotropic Gaussian Prototype Perturbation (IGPP) typically over-perturbs discriminative dimensions and struggles to balance the clipping threshold with representation fidelity. In this paper, we propose VPDR, a client-side privacy plug-in that seamlessly integrates into existing ProtoPFLs. Motivated by the observation that dimension-wise class variance reflects discriminability, we introduce Variance-adaptive Prototype Perturbation (VPP), which allocates less noise to discriminative subspaces, preserving semantic separability while ensuring privacy. We further develop Distillation-guided Clipping Regularization (DCR), which enables feature norms to adaptively concentrate near the predefined clipping threshold while maintaining prediction consistency. Theoretical analysis shows that our groupwise mechanism provides privacy guarantees no weaker than the isotropic baseline under the same privacy constraints. Extensive experiments on multi-domain benchmarks demonstrate that VPDR achieves a superior privacy-utility trade-off, outperforming IGPP in personalized federated fine-tuning without sacrificing robustness against realistic attacks.

Related papers

• Private Direct Preference Optimization for LLM Alignment [77.56374641532715]
  Direct preference optimization (DPO) is now a standard method for aligning large language models (LLMs) using human preference data.<n>In this paper, we formalize preference privacy, a label-DP-style privacy notion for DPO.<n>We then design PrivDPO, a DPO variant that enforces preference privacy while remaining compatible with large-scale LLM training.
  arXiv  Detail & Related papers  (2026-08-05T16:51:02Z)
• PAC-DP: Personalized Adaptive Clipping for Differentially Private Federated Learning [16.9867659381048]
  PAC-DP is a personalized Adaptive Clipping framework for federated learning.<n>We show that PAC-DP surpasses conventional fixed-threshold approaches under matched privacy budgets.
  arXiv  Detail & Related papers  (2026-03-25T07:06:33Z)
• Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions [89.52532304099522]
  Federated Learning (FL) enables clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy.<n> gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation.<n>These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework.<n>We propose Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping.
  arXiv  Detail & Related papers  (2026-03-24T17:39:09Z)
• Machine Learning with Privacy for Protected Attributes [56.44253915927481]
  We refine the definition of differential privacy (DP) to create a more general and flexible framework that we call feature differential privacy (FDP)<n>Our definition is simulation-based and allows for both addition/removal and replacement variants of privacy, and can handle arbitrary separation of protected and non-protected features.<n>We apply our framework to various machine learning tasks and show that it can significantly improve the utility of DP-trained models when public features are available.
  arXiv  Detail & Related papers  (2025-06-24T17:53:28Z)
• ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning [22.85986751447643]
  A common strategy involves integrating Byzantine-resilient aggregation rules with differential privacy mechanisms.<n>We propose ImprovDML, that achieves model accuracy while simultaneously ensuring privacy preservation.<n>We demonstrate that it enables an improved trade-off between model accuracy and differential privacy.
  arXiv  Detail & Related papers  (2025-06-18T06:53:52Z)
• Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models [12.406403248205285]
  Federated prompt personalization (FPP) is developed to address data heterogeneity and local overfitting.<n>We propose SecFPP, a secure FPP protocol harmonizing personalization, and privacy guarantees.<n>We show SecFPP significantly outperforms both non-private and privacy-preserving baselines.
  arXiv  Detail & Related papers  (2025-05-28T15:09:56Z)
• DPZV: Elevating the Tradeoff between Privacy and Utility in Zeroth-Order Vertical Federated Learning [9.302691218735406]
  We propose DPZV, the first ZO optimization framework for Vertical Federated Learning (VFL) that achieves tunable differential privacy with performance guarantees.<n>We conduct a comprehensive theoretical analysis showing that DPZV matches the convergence rate of first-order optimization methods while satisfying formal ($epsilon, delta$)-DP guarantees.<n>Experiments on image and language benchmarks demonstrate that DPZV outperforms several baselines in terms of accuracy under a wide range of privacy constraints.
  arXiv  Detail & Related papers  (2025-02-27T22:07:16Z)
• Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning [9.39508697970812]
  We propose a simple fix to the L1 Regularization, by employing adaptive penalties to different parameters. Our key idea behind PAST is to promote sparsity in parameters that significantly contribute to privacy leakage. Using PAST, the network shrinks the loss gap between members and non-members, leading to strong resistance to privacy attacks.
  arXiv  Detail & Related papers  (2024-10-09T12:13:49Z)
• Convergent Differential Privacy Analysis for General Federated Learning: the $f$-DP Perspective [57.35402286842029]
  Federated learning (FL) is an efficient collaborative training paradigm with a focus on local privacy. differential privacy (DP) is a classical approach to capture and ensure the reliability of private protections.
  arXiv  Detail & Related papers  (2024-08-28T08:22:21Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• RDP-GAN: A R\'enyi-Differential Privacy based Generative Adversarial Network [75.81653258081435]
  Generative adversarial network (GAN) has attracted increasing attention recently owing to its impressive ability to generate realistic samples with high privacy protection. However, when GANs are applied on sensitive or private training examples, such as medical or financial records, it is still probable to divulge individuals' sensitive and private information. We propose a R'enyi-differentially private-GAN (RDP-GAN), which achieves differential privacy (DP) in a GAN by carefully adding random noises on the value of the loss function during training.
  arXiv  Detail & Related papers  (2020-07-04T09:51:02Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.