An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning

• URL: http://arxiv.org/abs/2608.15153v1
• Date: Sat, 15 Aug 2026 10:10:34 GMT
• Title: An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning
• Abstract summary: This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning.<n>Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%.<n>It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy.
• Score: 1.5876034680722233
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differentially private federated learning must balance privacy protection against model accuracy and training efficiency. Static gradient clipping applies a fixed threshold throughout training and across model layers, which can cause excessive clipping when the threshold is too small or unnecessarily large noise when it is too large. This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning with secure aggregation. At each communication round, every client determines a separate clipping threshold for each model layer from the median of its per-sample gradient norms. The resulting layer-wise thresholds adapt to the evolving gradient distributions and calibrate the Laplace noise added before the updates are encoded and secret-shared among intermediate aggregation servers. We evaluate the proposed mechanism on a federated regression task in terms of efficiency, accuracy, privacy, convergence, clipping norm, and noise magnitude. Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%, leading to improved training efficiency. It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy. To attain R2 = 0.99, the adaptive mechanism operates with a privacy budget of approximately epsilon = 0.1, compared with epsilon = 0.4 for static DDP-SA, thus providing stronger privacy protection and achieving stronger privacy guarantees. These results demonstrate that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.

Related papers

• An Adaptive Differentially Private Federated Learning Framework with Bi-level Optimization [10.218291445871435]
  Federated learning enables collaborative model training across distributed clients while preserving data privacy.<n>In practical deployments, device heterogeneity, non-independent, and identically distributed (Non-IID) data often lead to highly unstable and biased gradient updates.<n>We propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings.
  arXiv  Detail & Related papers  (2026-02-06T16:27:33Z)
• Local Layer-wise Differential Privacy in Federated Learning [9.065322387043544]
  Federated Learning (FL) enables collaborative model training without direct data sharing, yet it remains vulnerable to privacy attacks such as model inversion and membership inference.<n>Existing differential privacy (DP) solutions for FL often inject noise uniformly across the entire model, degrading utility while providing suboptimal privacy-utility tradeoffs.<n>We propose LaDP, a novel layer-wise adaptive noise injection mechanism for FL that optimize privacy protection while preserving model accuracy.
  arXiv  Detail & Related papers  (2026-01-05T02:23:31Z)
• PLRV-O: Advancing Differentially Private Deep Learning via Privacy Loss Random Variable Optimization [16.48480873870065]
  We introduce PLRV-O, a framework that defines a broad search space of parameterized DP-SGD noise distributions.<n>We show that PLRV-O substantially improves utility under strict privacy constraints.
  arXiv  Detail & Related papers  (2025-09-08T01:06:45Z)
• Efficient Differentially Private Fine-Tuning of LLMs via Reinforcement Learning [0.9861588522527782]
  We present RLDP, the first framework to cast DP optimization itself as a closed-loop control problem amenable to modern deep reinforcement learning (RL)<n>Across more than 1,600 experiments on GPT2-small, Llama-1B, Llama-3B, and Mistral-7B, RLDP delivers perplexity reductions of 1.3-3.0.5% and an average 5.6% downstream utility gain.
  arXiv  Detail & Related papers  (2025-07-30T10:46:53Z)
• AdaDPIGU: Differentially Private SGD with Adaptive Clipping and Importance-Based Gradient Updates for Deep Neural Networks [1.7265013728931]
  We propose a new differentially private SGD framework with importance-based gradient updates tailored for deep neural networks.<n>AdaDPIGU satisfies $(varepsilon, delta)$-differential privacy and retains convergence guarantees.<n>On MNIST, our method achieves a test accuracy of 99.12% under a privacy budget of $epsilon = 8$, nearly matching the non-private model.
  arXiv  Detail & Related papers  (2025-07-09T03:53:03Z)
• DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction [57.83978915843095]
  This paper introduces DiSK, a novel framework designed to significantly enhance the performance of differentially private gradients.<n>To ensure practicality for large-scale training, we simplify the Kalman filtering process, minimizing its memory and computational demands.
  arXiv  Detail & Related papers  (2024-10-04T19:30:39Z)
• Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning [86.08285033925597]
  This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of federated learning. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
  arXiv  Detail & Related papers  (2023-03-07T22:52:40Z)
• On the Interaction Between Differential Privacy and Gradient Compression in Deep Learning [55.22219308265945]
  We study how the Gaussian mechanism for differential privacy and gradient compression jointly impact test accuracy in deep learning. We observe while gradient compression generally has a negative impact on test accuracy in non-private training, it can sometimes improve test accuracy in differentially private training.
  arXiv  Detail & Related papers  (2022-11-01T20:28:45Z)
• An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition [51.232523987916636]
  Differential privacy (DP) is one data protection avenue to safeguard user information used for training deep models by imposing noisy distortion on privacy data. In this work, we extend PATE learning to work with dynamic patterns, namely speech, and perform one very first experimental study on ASR to avoid acoustic data leakage.
  arXiv  Detail & Related papers  (2022-10-11T16:55:54Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.